Tuning method and system for multimedia sound effect, electronic equipment and storage medium

Abstract

The invention discloses a multimedia sound effect method and system, electronic equipment and a storage medium, and the method comprises the steps: carrying out the conversion and preprocessing of an obtained original dual-channel audio signal, and obtaining standard audio data; framing the standard audio data, and performing feature extraction to obtain a feature audio data set; analyzing the characteristic audio data set in different dimensions, determining a core parameter set and an extended parameter set, and inputting the core parameter set and the extended parameter set into a preset optimization model for optimization to obtain a tuning parameter set; and tuning the dual-channel audio signal according to the tuning parameter set. According to the invention, the dual-channel audio signals are collected, framing and feature extraction are carried out on the dual-channel audio signals, multi-dimensional analysis is carried out based on feature audio data, parameters of different dimensions are quantized, the multimedia sound effect is adjusted from different dimensions, manual repeated audition feedback is reduced, and the tuning efficiency is improved. The embodiment of the invention can be widely applied to the technical field of audio processing.

Description

Technical Field

[0001] The present invention relates to the field of audio processing technology, and in particular to a method, system, electronic device and storage medium for tuning multimedia sound effects. Background Art

[0002] With the development of software technology, users in the automotive field have put forward higher requirements for the auditory experience of multimedia sound effects in the cockpit. In terms of hardware, car cockpits are equipped with more advanced sound effect algorithms and high-quality speakers to meet users' higher requirements for multimedia sound effects. However, multimedia sound effects not only rely on hardware configuration, but also require parameter adjustment based on the song data, hardware configuration and cockpit environment, that is, sound tuning. Traditional tuning methods rely on engineers to repeatedly listen in a real car environment and adjust parameters based on experience feedback, which is inefficient. Summary of the Invention

[0003] The main purpose of the embodiments of the present invention is to provide a method, system, electronic device and storage medium for tuning multimedia sound effects, which can improve tuning efficiency.

[0004] To achieve the above objectives, an embodiment of the present invention provides a method for tuning multimedia sound effects, the method comprising:

[0005] Performing signal conversion on the original audio signal to determine original audio data, and preprocessing the original audio data to obtain standard audio data; wherein the original audio signal includes a two-way audio signal;

[0006] Performing frame processing on the standard audio data according to a preset step size to obtain a frame data set, and performing feature extraction on the frame data set to obtain a feature audio data set;

[0007] Performing a first analysis on the feature audio data set to obtain a core parameter set, performing a second analysis on the feature audio data set to obtain an extended parameter set, and determining an audio parameter set based on the core parameter set and the extended parameters;

[0008] The audio parameter set is optimized according to a preset optimization model to obtain a tuning parameter set, and a tuning operation is performed on the original audio signal according to the tuning parameter set.

[0009] In some embodiments, preprocessing the original audio data to obtain standard audio data specifically includes:

[0010] Performing frame processing on the original audio data according to a preset step size to obtain an original frame data set, and performing spectrum analysis on the original frame data set to determine a spectrum feature data set;

[0011] Performing denoising processing on the original frame dataset according to the spectral feature dataset to determine a denoised dataset, and performing reconstruction processing based on the denoised dataset to obtain reconstructed audio data;

[0012] The reconstructed audio data is smoothed to obtain denoised audio data, and the denoised audio data is normalized to obtain the standard audio data.

[0013] In some embodiments, extracting features from the framed data set to obtain a feature audio data set specifically includes:

[0014] Performing a first transformation on the frame data set to determine a first spectrum data set; extracting the first spectrum data set to obtain a Mel spectrum data set;

[0015] Performing a second transform process on the first spectrum data set to determine a cepstral coefficient set, and performing a filtering process on the first spectrum data set according to a preset filter to determine a time domain envelope feature set;

[0016] The feature audio dataset is determined according to the Mel-spectrogram dataset, the cepstral coefficient set, and the time-domain envelope feature set.

[0017] In some embodiments, performing a first analysis on the feature audio dataset to obtain a core parameter set specifically includes:

[0018] Dividing the characteristic audio data set according to preset frequency bands to determine a frequency band data set, identifying the frequency band data set to determine abnormal frequency point information; determining audio equalization parameters based on the abnormal frequency point information and preset rules;

[0019] Performing a frequency sweep analysis on the characteristic audio data set to determine group delay information and frequency band phase difference information; performing calculations based on the frequency band phase difference information to determine phase consistency information; and performing interference detection on the characteristic audio data set to determine time delay deviation information; and determining time domain feature information based on the group delay information, the phase consistency information, and the time delay deviation information;

[0020] Performing calculation and analysis based on a preset excitation signal and the characteristic audio data set to determine the audio reverberation time, performing evaluation based on a preset reflection threshold and the audio reverberation time to determine audio spatial information; performing density calculation on the characteristic audio data set to determine audio reverberation density information; and determining audio spatial parameters based on the audio reverberation time, the audio spatial information, and the audio reverberation density information.

[0021] The core parameter set is determined according to the audio equalization parameter, the time domain feature information, and the audio space information.

[0022] In some embodiments, performing a second analysis on the feature audio data set to obtain an extended parameter set specifically includes:

[0023] Calculating based on a preset compression threshold and the characteristic audio data to determine a threshold adjustment parameter;

[0024] Testing the characteristic audio data set according to a preset test signal to determine audio harmonic information, and analyzing the audio harmonic information and a preset algorithm to determine harmonic enhancement parameters;

[0025] Performing sound field analysis on the characteristic audio data set to determine audio sound field parameters, and determining stereo field parameters based on the audio sound field parameters and preset target sound field parameters;

[0026] The extended parameter set is determined according to the threshold adjustment parameter, the harmonic enhancement parameter, and the stereo field parameter.

[0027] In some embodiments, optimizing the audio parameter set according to a preset optimization model to obtain a tuning parameter set specifically includes:

[0028] Inputting the audio parameter set data into the preset optimization model for calculation to determine a reward estimate; and comparing the reward estimate with a preset evaluation threshold;

[0029] If the reward estimate is less than the preset evaluation threshold, adjusting the audio parameter set according to a preset adjustment strategy to determine an adjusted parameter set;

[0030] If the reward estimate is greater than or equal to the preset evaluation threshold, using the audio parameter set as the adjustment parameter set;

[0031] The adjustment parameter set is restricted according to preset constraints to determine the tuning parameter set.

[0032] In some embodiments, the method further comprises:

[0033] Determining objective audio parameters based on the comparison and verification of the tuned audio signal and the original audio signal; wherein the objective audio parameters include the stability and fluctuation range of the audio signal;

[0034] performing a correlation calculation based on the acquired audio score information and the tuning parameter set to determine a correlation coefficient; wherein the audio score information is determined by manually scoring the tuned audio signal;

[0035] Parameters of the preset optimization model are adjusted according to the correlation coefficient and the audio objective parameter.

[0036] To achieve the above objectives, another aspect of an embodiment of the present invention provides a multimedia sound effect tuning system, the system comprising:

[0037] The first module is configured to perform signal conversion on an original audio signal, determine original audio data, and pre-process the original audio data to obtain standard audio data; wherein the original audio signal includes a two-way audio signal;

[0038] The second module is used to perform frame processing on the standard audio data according to a preset step size to obtain a frame data set, and perform feature extraction on the frame data set to obtain a feature audio data set;

[0039] A third module is configured to perform a first analysis on the feature audio data set to obtain a core parameter set, perform a second analysis on the feature audio data set to obtain an extended parameter set, and determine an audio parameter set based on the core parameter set and the extended parameters;

[0040] The fourth module is configured to optimize the audio parameter set according to a preset optimization model to obtain a tuning parameter set, and perform a tuning operation on the original audio signal according to the tuning parameter set.

[0041] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned method when executing the computer program.

[0042] To achieve the above objectives, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0043] The implementation of the embodiments of the present invention includes the following beneficial effects: This embodiment provides a method, system, electronic device, and storage medium for tuning multimedia sound effects. The scheme converts the acquired original two-way audio signal and pre-processes the converted two-way audio signal to obtain standard audio data; frames the standard audio data and extracts features from the framed audio data to obtain a feature audio data set; then, analyzes the feature audio data set in different dimensions to determine a core parameter set and an extended parameter set of the feature audio data set, and inputs the core parameter set and the extended parameter set as audio parameter sets into a preset optimization model for optimization to obtain a corresponding tuning parameter set; finally, tunes the original two-way audio signal according to the obtained tuning parameter set. By collecting the two-way audio signal for tuning, noise interference is reduced; the two-way audio signal is framed, features are extracted, and multiple dimensions of analysis are performed based on the feature audio data to quantify parameters of different dimensions of the feature audio, optimize the parameters of different dimensions to obtain tuning parameters, and adjust the multimedia sound effects from different dimensions, reducing manual repeated audition feedback and improving tuning efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a schematic flow chart of the steps of a method for tuning multimedia sound effects provided by an embodiment of the present invention;

[0045] Figure 2 This is a flowchart of the steps for obtaining standard audio data in a method for tuning multimedia sound effects provided by an embodiment of the present invention;

[0046] Figure 3 This is a flowchart of the steps for obtaining characteristic audio data in a method for tuning multimedia sound effects provided by an embodiment of the present invention;

[0047] Figure 4 This is a flowchart of steps for obtaining a core parameter set in a method for tuning multimedia sound effects provided by an embodiment of the present invention;

[0048] Figure 5 This is a flowchart of the steps for obtaining an extended parameter set in a method for tuning multimedia sound effects provided by an embodiment of the present invention;

[0049] Figure 6 This is a flowchart of the steps for obtaining a tuning parameter set in a multimedia sound effect tuning method provided by an embodiment of the present invention;

[0050] Figure 7 This is a flowchart of the steps for dynamic fine-tuning in a method for tuning multimedia sound effects provided by an embodiment of the present invention;

[0051] Figure 8This is a structural block diagram of an implementation environment in a specific embodiment provided by an embodiment of the present invention;

[0052] Figure 9 This is a schematic diagram of the software framework of an intelligent tuning system in a specific embodiment provided by an embodiment of the present invention;

[0053] Figure 10 This is a flowchart of the steps for the AI ​​Agent to output tuning parameters in a specific embodiment provided by an embodiment of the present invention;

[0054] Figure 11 This is a schematic diagram of the steps of the sound effect algorithm of the equalizer in a specific embodiment provided by an embodiment of the present invention;

[0055] Figure 12 This is a structural block diagram of a multimedia sound effect tuning system provided by an embodiment of the present invention;

[0056] Figure 13 This is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention;

[0057] Among them, 1 is the first microphone; 2 is the second microphone; 3 is the cockpit speaker; 4 is the tuning module, 5 is the digital-to-analog converter; 6 is the device interface and communication network. DETAILED DESCRIPTION

[0058] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are provided for ease of description only and do not limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted based on the understanding of those skilled in the art.

[0059] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0060] In the following description, the terms "first\second\third" are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein.

[0061] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present invention have the same meanings as those commonly understood by those skilled in the art to which the present invention pertains. The terms used in the embodiments of the present invention are for the purpose of describing the embodiments of the present invention only and are not intended to limit the present invention.

[0062] Figure 1 This is an optional flowchart of a method for tuning multimedia sound effects provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S104.

[0063] Step S101, performing signal conversion on the original audio signal, determining original audio data, and preprocessing the original audio data to obtain standard audio data; wherein the original audio signal includes a two-way audio signal;

[0064] Step S102: performing frame processing on the standard audio data according to a preset step size to obtain a frame data set, and performing feature extraction on the frame data set to obtain a feature audio data set;

[0065] Step S103: performing a first analysis on the feature audio data set to obtain a core parameter set, performing a second analysis on the feature audio data set to obtain an extended parameter set, and determining an audio parameter set based on the core parameter set and the extended parameters;

[0066] Step S104 : Optimizing the audio parameter set according to a preset optimization model to obtain a tuning parameter set, and performing a tuning operation on the original audio signal according to the tuning parameter set.

[0067] In steps S101 to S104 shown in the embodiment of the present application, a dual-channel microphone is set in the car cabin to simulate the two ears of a human, and the audio of the multimedia playing songs in the cabin is collected to obtain the original audio signal; the original audio signal in the analog signal format collected by the microphone is converted into a digital signal through the ADC module to obtain the original audio data; then, the background noise of the original audio data is suppressed by using a noise reduction algorithm or a constructed noise reduction model, and the audio data after noise suppression is normalized to obtain standard audio data; then, the obtained standard audio data is framed to obtain multiple short-term stable audio clips; feature extraction is performed on the obtained audio clips to obtain Mel spectrum data and Mel spectrum cepstral coefficients in the audio data to simulate the nonlinear auditory characteristics of the human ear and compress the high-frequency information in the audio data; at the same time, time domain envelope extraction is performed on the audio clips to capture the macroscopic characteristics of the energy changing with time in the original audio signal; the extracted Mel spectrum data and Mel spectrum cepstral coefficients, as well as the time domain envelope data, are used as feature information of the audio signal to obtain a feature audio data set, that is, audio frequency data with feature information; , perform audio analysis on the feature audio dataset, perform core dimension analysis in turn to obtain a core parameter set, and perform extended parameter supplementation to obtain an extended parameter set; for core dimension analysis, perform frequency band energy analysis on the feature audio dataset in turn, identify abnormal frequency points that need to be adjusted, and determine the adjustment parameters; then measure the group delay information, phase consistency, and delay deviation caused by multipath interference of the feature audio dataset to determine the time domain characteristics of the feature audio dataset; finally, analyze the reverberation time and reverberation density in the feature audio dataset, evaluate the spatial sense information of the feature audio dataset, and determine the spatial parameters of the feature audio dataset; for extended parameter supplementation, analyze the compression ratio in the feature audio dataset, judge whether the dynamic range of the feature audio dataset is normal, determine whether to adjust the parameters of the feature audio dataset accordingly, and improve the sound quality of the feature audio dataset; then, perform harmonic analysis on the feature audio dataset to judge whether it is distorted and determine the corresponding optimized harmonic parameters; finally, perform sound image width and other analyses on the feature audio dataset, establish a corresponding stereo field, judge whether it meets the sound field parameters of the target audio, and then determine the corresponding sound field parameters. The obtained parameter set is input into the trained optimization model for further optimization to obtain the final tuning parameter set, which is then sent to the sound equalization module in the cockpit multimedia. The sound equalization module tunes the output song audio according to the tuning parameter set.

[0068] In some embodiments, step S101 may utilize dual microphones to capture multimedia audio in real time for adjustment, with a sampling rate of no less than 48kHz and a bit depth of 24 bits to preserve high-frequency details. Alternatively, real-time streaming and offline file import may be used, supporting WAV / FLAC formats to accommodate diverse scenarios, without limitation.

[0069] See also Figure 2 In some embodiments, step S101 may include but is not limited to steps S201 to S203:

[0070] Step S201, performing frame processing on the original audio data according to a preset step size to obtain an original frame data set, and performing spectrum analysis on the original frame data set to determine a spectrum feature data set;

[0071] Step S202, performing denoising processing on the original frame dataset according to the spectral feature dataset to determine a denoised dataset, and performing reconstruction processing based on the denoised dataset to obtain reconstructed audio data;

[0072] Step S203: smoothing the reconstructed audio data to obtain denoised audio data, and normalizing the denoised audio data to obtain standard audio data.

[0073] In step S201 of some embodiments, the two-way audio signal collected by the microphone is converted from an analog signal format to a two-way audio signal in a digital signal format through an ADC module. The two-way audio signal is a continuous, fluctuating audio signal. In order to facilitate time-frequency analysis, the time domain and frequency domain features are extracted, and the continuous audio signal is divided into short-time stationary segments according to a preset frame length and frame shift. In this embodiment, the frame length is set to 25ms and the frame shift is set to 10ms. Then, the divided frame segments are converted into the frequency domain to obtain the spectral feature data of the audio, for example, by using Fourier transform.

[0074] In step S202 of some embodiments, the obtained spectral feature data is subjected to denoising to suppress background noise in the original audio data, thereby obtaining denoised spectral data. In this embodiment, spectral feature data is denoised using spectral subtraction or by constructing a deep learning model to obtain denoised spectral data. Due to the denoising process, the phase information of the denoised spectral data may differ from the phase information of the original audio data. To avoid losing the time-domain and frequency-domain feature information in the audio, the denoised spectral data is phase-reconstructed using the phase information of the original audio data, and an inverse transform is performed to obtain reconstructed continuous audio data.

[0075] In step S203 of some embodiments, the reconstructed audio data is smoothed to restore the actual audio data as much as possible and reduce the frequency mutations in the audio data to obtain denoised audio data; finally, the denoised audio data is normalized to obtain standard audio data; in this embodiment, the level of the denoised audio data is normalized to -24UFS.

[0076] See also Figure 3 In some embodiments, step S102 may include but is not limited to steps S301 to S303:

[0077] Step S301, performing a first transformation on the frame data set to determine a first spectrum data set; extracting the first spectrum data set to obtain a Mel spectrum data set;

[0078] Step S302, performing a second transformation process on the first spectrum data set to determine a cepstral coefficient set, and performing filtering process on the first spectrum data set according to a preset filter to determine a time domain envelope feature set;

[0079] Step S303 : determining a feature audio data set according to the mel-spectrogram data set, the cepstral coefficient set, and the time-domain envelope feature set.

[0080] In step S301 of some embodiments, the preprocessed continuous audio data is segmented according to a preset frame length and frame shift to obtain short-time stationary segments; the short-time stationary segments obtained by segmentation are subjected to short-time Fourier transform to obtain corresponding spectral data, so as to extract the Mel spectrum from the audio data, and the nonlinear auditory characteristics of the ear are simulated through the Mel spectrum to compress high-frequency information to improve the tuning effect of subsequent multimedia sound effects; the obtained spectral data is mapped using a Mel filter, and energy weighting and logarithmic compression are performed to obtain corresponding Mel spectrum data.

[0081] In step S302 of some embodiments, after obtaining the mel spectrum data, the mel spectrum data is transformed using a discrete cosine transform method, and the transformed data is energy-weighted and logarithmically compressed to achieve dynamic feature expansion and obtain mel-frequency cepstral coefficients corresponding to the audio data; the mel spectrum data and the mel-frequency cepstral coefficients are used to simulate the auditory characteristics of the human ear; then, the mel spectrum data is filtered using a low-pass filter to extract time domain envelope information from the spectrum data to simulate the human ear's perception limit of amplitude changes; in this embodiment, the cutoff frequency of the low-pass filter is set to 20 Hz.

[0082] In step S303 of some embodiments, the previously obtained mel spectrum data, mel frequency cepstral coefficients, and time domain envelope information are summarized and sorted to obtain a characteristic audio data set of the audio signal for subsequent audio parameter analysis and extraction to determine the adjustment parameters of the two-way audio data.

[0083] See also Figure 4 In some embodiments, step S103 may include but is not limited to steps S401 to S404:

[0084] Step S401: dividing the characteristic audio data set according to preset frequency bands to determine a frequency band data set, identifying the frequency band data set to determine abnormal frequency point information; and determining audio equalization parameters based on the abnormal frequency point information and preset rules;

[0085] Step S402: Perform a frequency sweep analysis on the characteristic audio data set to determine group delay information and frequency band phase difference information; perform calculations based on the frequency band phase difference information to determine phase consistency information; perform interference detection on the characteristic audio data set to determine time delay deviation information; and determine time domain feature information based on the group delay information, phase consistency information, and time delay deviation information.

[0086] Step S403: Performing calculations and analysis based on the preset excitation signal and the characteristic audio data set to determine the audio reverberation time, performing an evaluation based on the preset reflection threshold and the audio reverberation time to determine the audio spatial information; performing density calculation on the characteristic audio data set to determine the audio reverberation density information; and determining the audio spatial parameters based on the audio reverberation time, the audio spatial information, and the audio reverberation density information.

[0087] Step S404: determining a core parameter set according to the audio equalization parameters, the time domain feature information, and the audio space information.

[0088] In step S401 of some embodiments, after obtaining the audio feature data, it is necessary to perform a multi-dimensional analysis on the audio feature data. First, the audio feature data equalization parameters are analyzed, and the energy distribution of the audio feature data is analyzed based on a certain frequency band to identify whether there are audio points in the audio feature data that need to be adjusted; before identification, the audio feature data is divided into high, medium and low frequency bands according to the set standard frequency band; then, energy proportion analysis is performed on different frequency bands to analyze whether the energy proportion of different frequency bands is a certain proportion, and to determine whether there is an abnormality in the frequency band; corresponding processing measures are taken according to the identified abnormal frequency band, and then corresponding adjustment parameters are determined; illustratively, audio data with a frequency range of 20Hz to 250Hz in the audio feature data is divided into a low frequency band; the energy proportion of the low frequency band is calculated to determine the proportion of the low frequency band energy in the total energy of the entire audio data; If the energy of the low frequency band is greater than 40%, it means that there is a low-frequency overload phenomenon in the low frequency band, and the audio signal in the low frequency band needs to be attenuated. The attenuation of the low frequency band is calculated according to the set processing strategy, and the corresponding attenuation gradient is set, etc. The calculated attenuation or gain is used as the audio equalization parameter of the audio data; in this embodiment, it is divided according to the standard frequency band: low frequency: 20-250Hz; intermediate frequency: 250-6kHz; high frequency: 6-20kHz; among them, the low frequency can be subdivided into subwoofer 20-60Hz, bass 60-200Hz, and intermediate low frequency 200-250Hz; the intermediate frequency can be subdivided into intermediate low frequency 250-500Hz, intermediate frequency 500-2kHz, and intermediate high frequency 2-6kHz; the high frequency can be subdivided into treble 6-10kHz and very treble 10-20kHz; the system sets the following table based on acoustic engineering experience for abnormal frequency point detection.

[0089] Table 1

[0090] Exception Type condition Solution Low frequency overload 20-200Hz energy share>40% Attenuation 60-200Hz frequency band Mid-frequency notch 500-2kHz energy share <25% Gain 800Hz-1.6kHz High-frequency distortion 6-10kHz energy drop> 15dB / octave Improve the Q value of 8-12kHz

[0091] In step S402 of some embodiments, a vector network analyzer is then used to output a continuous frequency sweep signal within the target frequency band for the obtained audio feature data, and a phase-frequency curve of the output signal is collected, and the sampling interval of the collection can be set to no more than 1 / 10 octave; group delay calculation is performed based on the collected phase-frequency curve and a preset calculation formula to obtain group delay data of the audio data; then, phase difference statistics of the entire frequency band are performed based on the collected phase-frequency curve, and a consistency score of the audio data is calculated based on the statistical phase difference and a preset calculation formula to determine the phase consistency information of the audio data; finally, impulse response peak detection is performed on the audio data based on the set pulse signal, and the maximum amplitude point of the impulse response is identified as the main peak of the audio data. A detection threshold is set to detect and extract the amplitude of the impulse response, and a secondary peak is determined as a reflection peak. The delay deviation information of the audio data under multipath interference is calculated based on the extracted main peak and reflection peak; the system summarizes and organizes the group delay data, phase consistency information, and delay deviation information obtained above to obtain time domain feature information of the audio data.

[0092] In some embodiments, in step S403, after obtaining the time-domain feature information of the audio data, a preset maximum-length sequence or a sine-sweep signal is used to perform sound field excitation on the audio data to capture the impulse response in the cabin space. An energy decay curve is calculated based on the captured impulse response, and the reverberation time is estimated based on the energy decay curve. In this embodiment, the -5 dB to -35 dB interval of the energy decay curve is located, and the slope of the energy decay curve in this interval is linearly fitted and extrapolated to -60 dB to obtain the audio reverberation time. The fitted data is then analyzed for the proportion of early reflection energy. Time windows are first divided according to the timing information, and energy statistics are performed on different stages to determine the proportion of early reflections. The early reflection proportion is compared with a preset threshold and analyzed in combination with the audio reverberation time segment to determine the spatial information provided by the current audio data. For example, when the audio reverberation time is between 0.2 s and 0.4 s and the early reflection proportion exceeds 25%, it is determined that the spatial sense of the current audio data is similar to that of a small recording studio and has good multimedia sound effects. In this embodiment, the spatial sense evaluation criteria are set as shown in the following table:

[0093] Table 2

[0094] Space Type Audio reverberation time Early reflection ratio threshold Small recording studio 0.2-0.4s ＞25％ Concert Hall 1.6-2.2s 15％-20％ church ＞3s ＜10％

[0095] Next, the filter group is used to decompose the audio data of different frequency bands and calculate the attenuation slope; the density index of the current audio data is calculated based on the calculated attenuation slope; the target audio attenuation curve is aligned with the standard acoustic space database, and perceptual weighting is performed to weight the corresponding frequency bands, adjust the overall reverberation density of the audio data, and then adjust the spatial sense information of the audio data; the audio reverberation time, audio spatial sense information and audio reverberation density information obtained above are summarized and sorted to obtain the audio spatial parameters of the audio data.

[0096] In step S404 of some embodiments, the obtained audio equalization parameters, time domain feature information, and audio space information are summarized and organized to obtain a core parameter set of audio data; the core parameter set can be used to adjust the two-way audio signal, thereby improving the multimedia sound effects in the cabin to a certain extent.

[0097] See also Figure 5 In some embodiments, step S103 may also include but is not limited to steps S501 to S504:

[0098] Step S501, calculating based on a preset compression threshold and characteristic audio data to determine a threshold adjustment parameter;

[0099] Step S502: testing the characteristic audio data set according to a preset test signal to determine audio harmonic information, and analyzing the audio harmonic information and a preset algorithm to determine harmonic enhancement parameters;

[0100] Step S503, performing sound field analysis on the characteristic audio data set to determine audio sound field parameters, and determining stereo field parameters based on the audio sound field parameters and preset target sound field parameters;

[0101] Step S504: determining an extended parameter set according to the threshold adjustment parameter, the harmonic enhancement parameter, and the stereo field parameter.

[0102] In step S501 of some embodiments, in order to further improve the tuning effect of multimedia sound effects, the system supplements the audio data with extended parameters and optimizes the tuning details of the multimedia sound effects; first, the relationship between the critical level value for starting the compressor and the amplitude of the audio signal is detected to determine whether to adjust the starting threshold of the compressor to suppress the compressor from over-adjusting the dynamic range of the audio data; in this embodiment, when the audio signal is too dynamic, if the set starting threshold is close to the signal average level of the audio signal, the starting threshold is increased and the compression ratio of the compressor is increased; when the audio signal is too dynamic, if the set starting threshold is close to the peak level of the audio signal, the compression ratio of the compressor is set to a negative compression ratio.

[0103] In step S502 of some embodiments, the system then sets a test signal, processes the audio data using the test signal and a digital filter, filters out the fundamental frequency component in the audio data, performs spectrum analysis and parameter calculation on the processed audio data, calculates the ratio of total harmonic distortion and noise in the audio data, and uses a negative feedback network for processing based on the ratio to obtain optimized harmonic enhancement parameters to improve the quality of the audio data and improve the stability of the audio data. In this embodiment, the parameter settings of the negative feedback network are shown in the following table:

[0104] Table 3

[0105] Feedback coefficient range Harmonic suppression effect Stability requirements 0.1-0.3 Moderate suppression (total harmonic distortion and noise contribution approximately equal to 0.1%) Phase margin ≥60° 0.3-0.5 Strong suppression (total harmonic distortion and noise ≤ 0.01%) Compensation capacitor is required to prevent self-excitation

[0106] In step S503 of some embodiments, finally, the audio data is subjected to sound and image width analysis, the correlation between the left and right channel data of the two-way audio data is calculated, the sound field data of the audio data is established using the two-way audio data, the sound field data of the audio data is matched with the sound field parameters of the target audio played through a specific scene, and the adjustment parameters corresponding to the sound field data of the current audio data are determined.

[0107] In step S504 of some embodiments, the threshold adjustment parameters, optimized harmonic enhancement parameters, and sound field adjustment parameters obtained above are summarized and sorted to obtain an extended parameter set, so as to optimize the tuning details of the multimedia sound effects.

[0108] See also Figure 6 In some embodiments, step S104 may also include but is not limited to steps S601 to S604:

[0109] Step S601: Input the audio parameter set data into a preset optimization model for calculation to determine a reward estimate; and compare the reward estimate with a preset evaluation threshold;

[0110] Step S602: If the estimated reward value is less than a preset evaluation threshold, the audio parameter set is adjusted according to a preset adjustment strategy to determine an adjusted parameter set;

[0111] Step S603: If the estimated reward value is greater than or equal to the preset evaluation threshold, the audio parameter set is used as an adjustment parameter set;

[0112] Step S604: performing restriction processing on the adjustment parameter set according to the preset constraint conditions to determine the tuning parameter set.

[0113] In step S601 of some embodiments, after obtaining the core parameter set and extended parameter set of the audio data, the core parameter set and the extended parameter set are input into a trained parameter optimization model for optimization processing. The parameter optimization model is obtained by iteratively training the constructed deep Q network model. During the training, the target audio features are used as the training reward function; the optimization model restores the audio data for the input core parameter set and the extended parameter set, extracts the spectral feature information therein, and calculates based on the spectral feature information to obtain the reward estimate corresponding to the current audio data, that is, the Q value; the parameter optimization model determines to perform corresponding optimization processing on the input core parameter set and the extended parameter set based on the preset threshold and the calculated reward estimate.

[0114] In step S602 of some embodiments, if the parameter optimization model determines that the estimated reward value is less than a set threshold, the parameter optimization model performs random exploration according to the set multiple optimization strategies, and adjusts the input core parameter set and extended parameter set accordingly according to the optimization strategy obtained from the exploration to obtain an adjusted parameter set.

[0115] In step S603 of some embodiments, if the parameter optimization model determines that the reward estimate is greater than or equal to a set threshold, the parameter optimization model takes the input core parameter set and extended parameter set as the optimal solution and outputs them as an adjusted parameter set.

[0116] In step S604 of some embodiments, in order to avoid excessive delay and audio mutation in the tuned audio data, corresponding delay control and parameter smoothing constraints are set to restrict the adjustment parameter set output by the parameter optimization model, and a first-order low-pass filter is applied to prevent mutations to obtain the final tuning parameter set.

[0117] See also Figure 7 In some embodiments, the multimedia sound effect tuning method provided in the embodiments of the present application may further include but is not limited to steps S701 to S703:

[0118] Step S701: comparing and verifying the tuned audio signal with the original audio signal to determine objective audio parameters; wherein the objective audio parameters include the stability and fluctuation range of the audio signal;

[0119] Step S702: performing a correlation calculation based on the acquired audio score information and the tuning parameter set to determine a correlation coefficient; wherein the audio score information is determined by manually scoring the tuned audio signal;

[0120] Step S703 : adjusting parameters of the preset optimization model according to the correlation coefficient and the audio objective parameters.

[0121] In step S701 of some embodiments, in order to improve the tuning applicability of different multimedia sound effects and meet the personalized needs of users, the system can obtain real-time feedback information from users in real time, and fine-tune the parameter optimization model in real time through the real-time feedback information from users to adapt to the personalized requirements of different users; the system continues to compare and verify the two-way audio data after tuning with the two-way audio data before tuning, detects the fluctuation range and audio stability of the audio data after tuning by the tuning parameter set, and uses the obtained fluctuation range and audio stability of the audio data as objective parameters of the audio.

[0122] In step S702 of some embodiments, the system then obtains the user's subjective evaluation of the tuning, and based on the difference between the user's subjective evaluation and the actual output of the system, the system fine-tunes the parameters of the parameter optimization model according to the difference to meet the user's personalized needs; in this embodiment, several groups of audio clips can be divided, and the user can score the "naturalness" of the divided groups of audio clips, and then the system calculates the correlation coefficient between the parameter optimization model and the manual "naturalness" score to determine whether the tuning effect of the tuning parameters output by the model is close to the user's needs, and then determine the amplitude of adjusting the parameters of the parameter optimization model.

[0123] In step S703 of some embodiments, the system performs online fine-tuning on the parameter optimization model according to the calculated objective parameters of the audio and the calculated correlation coefficient, thereby improving the flexibility of tuning of the multimedia sound effects.

[0124] The following describes the solution of the embodiment of the present invention in detail with reference to specific application examples:

[0125] The embodiment of the present invention provides a method for tuning multimedia sound effects, which is applied to Figure 8 In the car cockpit environment shown in the figure, the multimedia audio played in the car cockpit environment is tuned in real time by building the AIAudio agent; in the car cockpit, such as Figure 9 As shown, the AIAudio agent is deployed on the PC side and uses dual MICs for recording (simulating human ears). The AIagent analyzes the audio file and provides fix parameters. The tuning parameters are delivered via USB / eTH. The parameters are the control parameters of the DSP sound effect algorithm for the audio. The DA passes the tuning parameters to the sound effect algorithm through the SOC and SIP. The DSP sound effect algorithm calibrates the actual audio playback; please refer to Figure 10, the speakers in the car cabin play audio, the AI ​​Audio agent includes an audio input and preprocessing module, an audio analysis and target comparison module, and a parameter adjustment and optimization module; the audio input and preprocessing module collects the audio signals played through dual microphones, and converts the audio signals collected by the microphones into audio data through an analog-to-digital converter; then, if the input is an audio file, the input audio file is converted into audio stream data through the corresponding processing module; then, the constructed deep learning model is used to reduce noise and standardize the input audio stream data to obtain normalized audio data; the obtained normalized audio data is framed to obtain several audio clips, and features are extracted for each audio clip to obtain Mel spectrum, Mel frequency cepstral coefficients and time domain envelope data as feature data of the audio stream data, and the feature data is used as the feature data of the audio stream data. The feature data is input into the audio analysis and target comparison module. The audio analysis and target comparison module first performs core analysis dimension processing on the input feature data, and extracts the EQ parameters, time domain characteristics and spatial parameters of the feature data in turn; then, the feature data is supplemented with extended parameters to obtain the corresponding dynamic range, harmonic distortion information and stereo field information; the obtained extended parameter supplement, core analysis dimension results and feature data are input into the parameter adjustment and optimization module, and the parameter adjustment and optimization module is set to adaptively adjust and optimize the input data according to the expert rule library and reinforcement learning model to obtain the corresponding adjustment parameters; through real-time feedback and iteration modules, A / B test comparison is performed, and the reinforcement learning model is fine-tuned online to continuously optimize the audio adjustment parameters to obtain the final correction parameters; AI Audioagent sends the correction parameters to the equalizer through the interface and communication network, and the equalizer performs tuning processing according to the built-in sound effect algorithm, as shown below. Figure 11 As shown, the tuned audio data is then converted into an analog signal through a digital-to-analog converter and played by the speakers in the cockpit to complete the tuning of the multimedia sound effects.

[0126] The implementation of the embodiments of the present invention includes the following beneficial effects: This embodiment provides a method, system, electronic device, and storage medium for tuning multimedia sound effects. The scheme converts the acquired original two-way audio signal and pre-processes the converted two-way audio signal to obtain standard audio data; frames the standard audio data and extracts features from the framed audio data to obtain a feature audio data set; then, analyzes the feature audio data set in different dimensions to determine a core parameter set and an extended parameter set of the feature audio data set, and inputs the core parameter set and the extended parameter set as audio parameter sets into a preset optimization model for optimization to obtain a corresponding tuning parameter set; finally, tunes the original two-way audio signal according to the obtained tuning parameter set. By collecting the two-way audio signal for tuning, noise interference is reduced; the two-way audio signal is framed, features are extracted, and multiple dimensions of analysis are performed based on the feature audio data to quantify parameters of different dimensions of the feature audio, optimize the parameters of different dimensions to obtain tuning parameters, and adjust the multimedia sound effects from different dimensions, reducing manual repeated audition feedback and improving tuning efficiency.

[0127] like Figure 12 As shown, an embodiment of the present invention further provides a multimedia sound effect tuning system, which can implement the above-mentioned multimedia sound effect tuning method, and the system includes:

[0128] The first module is configured to perform signal conversion on an original audio signal, determine original audio data, and pre-process the original audio data to obtain standard audio data; wherein the original audio signal includes a two-way audio signal;

[0129] The second module is used to perform frame processing on the standard audio data according to a preset step size to obtain a frame data set, and perform feature extraction on the frame data set to obtain a feature audio data set;

[0130] A third module is configured to perform a first analysis on the feature audio data set to obtain a core parameter set, perform a second analysis on the feature audio data set to obtain an extended parameter set, and determine an audio parameter set based on the core parameter set and the extended parameters;

[0131] The fourth module is configured to optimize the audio parameter set according to a preset optimization model to obtain a tuning parameter set, and perform a tuning operation on the original audio signal according to the tuning parameter set.

[0132] It can be seen that the contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0133] The present application also provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the aforementioned multimedia sound effect tuning method. The electronic device can be any smart terminal, such as a tablet computer or an in-vehicle computer.

[0134] It can be understood that the contents of the above method embodiments are applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0135] See also Figure 13 , Figure 13 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0136] The processor 1301 can be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0137] The memory 1302 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1302 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1302 and is called by the processor 1301 to execute a multimedia sound effect tuning method according to the embodiments of this application.

[0138] Input / output interface 1303, used to implement information input and output;

[0139] Communication interface 1304, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0140] Bus 1305 , which transmits information between various components of the device (e.g., processor 1301 , memory 1302 , input / output interface 1303 , and communication interface 1304 );

[0141] The processor 1301 , the memory 1302 , the input / output interface 1303 and the communication interface 1304 are connected to each other in communication within the device via a bus 1305 .

[0142] Among them, the memory is a non-transient computer-readable storage medium that can be used to store non-transient software programs and non-transient computer executable programs. The memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory optionally includes a remote memory remotely arranged relative to the processor, and these remote memories can be connected to the processor via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0143] In addition, the embodiments of the present application further disclose a computer program product or computer program, which is stored in a computer-readable storage medium. The processor of a computer device can read the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device performs the above-mentioned method. Similarly, the contents of the above-mentioned method embodiment are all applicable to the present storage medium embodiment, and the functions specifically implemented by the present storage medium embodiment are the same as those of the above-mentioned method embodiment, and the beneficial effects achieved are also the same as those achieved by the above-mentioned method embodiment.

[0144] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned method for tuning multimedia sound effects.

[0145] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiment, the functions specifically implemented by the present storage medium embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0146] It is understood that all or some steps, systems in the disclosed method above can be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components can be implemented as software by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application specific integrated circuit. Such software can be distributed on a computer-readable medium, and the computer-readable medium can include computer storage media (or non-transitory media) and communication media (or temporary media). As known to those of ordinary skill in the art, the term computer storage medium is included in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data) and is volatile and non-volatile, removable and non-removable media. Computer storage media includes but is not limited to RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, magnetic tape, disk storage or other magnetic storage device, or can be used to store desired information and any other medium that can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0147] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A method for tuning multimedia sound effects, characterized in that: The method comprises: Performing signal conversion on the original audio signal to determine original audio data, and preprocessing the original audio data to obtain standard audio data; wherein the original audio signal includes a two-way audio signal; Performing frame processing on the standard audio data according to a preset step size to obtain a frame data set, and performing feature extraction on the frame data set to obtain a feature audio data set; Performing a first analysis on the feature audio data set to obtain a core parameter set, performing a second analysis on the feature audio data set to obtain an extended parameter set, and determining an audio parameter set based on the core parameter set and the extended parameters; The audio parameter set is optimized according to a preset optimization model to obtain a tuning parameter set, and a tuning operation is performed on the original audio signal according to the tuning parameter set.

2. The method according to claim 1, characterized in that The preprocessing of the original audio data to obtain standard audio data specifically includes: Performing frame processing on the original audio data according to a preset step size to obtain an original frame data set, and performing spectrum analysis on the original frame data set to determine a spectrum feature data set; Performing denoising processing on the original frame dataset according to the spectral feature dataset to determine a denoised dataset, and performing reconstruction processing based on the denoised dataset to obtain reconstructed audio data; The reconstructed audio data is smoothed to obtain denoised audio data, and the denoised audio data is normalized to obtain the standard audio data.

3. The method according to claim 1, characterized in that The feature extraction of the frame data set to obtain a feature audio data set specifically includes: Performing a first transformation on the frame data set to determine a first spectrum data set; extracting the first spectrum data set to obtain a Mel spectrum data set; Performing a second transform process on the first spectrum data set to determine a cepstral coefficient set, and performing a filtering process on the first spectrum data set according to a preset filter to determine a time domain envelope feature set; The feature audio dataset is determined according to the Mel-spectrogram dataset, the cepstral coefficient set, and the time-domain envelope feature set.

4. The method according to claim 1, wherein The first analysis of the feature audio data set to obtain a core parameter set specifically includes: Dividing the characteristic audio data set according to preset frequency bands to determine a frequency band data set, identifying the frequency band data set to determine abnormal frequency point information; determining audio equalization parameters based on the abnormal frequency point information and preset rules; Performing a frequency sweep analysis on the characteristic audio data set to determine group delay information and frequency band phase difference information; performing calculations based on the frequency band phase difference information to determine phase consistency information; and performing interference detection on the characteristic audio data set to determine time delay deviation information; and determining time domain feature information based on the group delay information, the phase consistency information, and the time delay deviation information; Performing calculation and analysis based on a preset excitation signal and the characteristic audio data set to determine the audio reverberation time, performing evaluation based on a preset reflection threshold and the audio reverberation time to determine audio spatial information; performing density calculation on the characteristic audio data set to determine audio reverberation density information; and determining audio spatial parameters based on the audio reverberation time, the audio spatial information, and the audio reverberation density information. The core parameter set is determined according to the audio equalization parameter, the time domain feature information, and the audio space information.

5. The method according to claim 1, wherein The second analysis of the feature audio data set to obtain an extended parameter set specifically includes: Calculating based on a preset compression threshold and the characteristic audio data to determine a threshold adjustment parameter; Testing the characteristic audio data set according to a preset test signal to determine audio harmonic information, and analyzing the audio harmonic information and a preset algorithm to determine harmonic enhancement parameters; Performing sound field analysis on the characteristic audio data set to determine audio sound field parameters, and determining stereo field parameters based on the audio sound field parameters and preset target sound field parameters; The extended parameter set is determined according to the threshold adjustment parameter, the harmonic enhancement parameter, and the stereo field parameter.

6. The method according to claim 1, characterized in that Optimizing the audio parameter set according to the preset optimization model to obtain the tuning parameter set specifically includes: Inputting the audio parameter set data into the preset optimization model for calculation to determine a reward estimate; and comparing the reward estimate with a preset evaluation threshold; If the reward estimate is less than the preset evaluation threshold, adjusting the audio parameter set according to a preset adjustment strategy to determine an adjusted parameter set; If the reward estimate is greater than or equal to the preset evaluation threshold, using the audio parameter set as the adjustment parameter set; The adjustment parameter set is restricted according to preset constraints to determine the tuning parameter set.

7. The method according to claim 1, characterized in that The method further comprises: Determining objective audio parameters based on the comparison and verification of the tuned audio signal and the original audio signal; wherein the objective audio parameters include the stability and fluctuation range of the audio signal; performing a correlation calculation based on the acquired audio score information and the tuning parameter set to determine a correlation coefficient; wherein the audio score information is determined by manually scoring the tuned audio signal; Parameters of the preset optimization model are adjusted according to the correlation coefficient and the audio objective parameter.

8. A multimedia sound effect tuning system, characterized in that: include: The first module is configured to perform signal conversion on an original audio signal, determine original audio data, and pre-process the original audio data to obtain standard audio data; wherein the original audio signal includes a two-way audio signal; The second module is used to perform frame processing on the standard audio data according to a preset step size to obtain a frame data set, and perform feature extraction on the frame data set to obtain a feature audio data set; A third module is configured to perform a first analysis on the feature audio data set to obtain a core parameter set, perform a second analysis on the feature audio data set to obtain an extended parameter set, and determine an audio parameter set based on the core parameter set and the extended parameters; The fourth module is configured to optimize the audio parameter set according to a preset optimization model to obtain a tuning parameter set, and perform a tuning operation on the original audio signal according to the tuning parameter set.

9. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The processor-executable program is configured to perform the method according to any one of claims 1 to 7 when executed by the processor.

Images