Multi-mode audio processing method, device, Soc chip and storage medium

Abstract

The present invention provides a multi-mode audio processing method, device, Soc chip and storage medium, the method comprising: performing frame processing on the input audio signal, detecting the effective signal and evaluating the channel quality, and obtaining the channel feature vector; marking the channel mode according to the channel feature vector, generating an audio data frame sequence with the channel mode mark, and judging whether there is a channel mode switch; if there is, processing the audio data frame sequence according to the channel feature vector and the channel mode mark, setting the cross-transition time window of the channel mode switch during the processing, and generating a processed data sequence with a transition mark; performing gradual processing on the signal amplitude of adjacent data frames according to the transition mark, smoothly connecting the audio signals before and after the switch within the cross-transition time window, and outputting a continuous audio signal. The present invention realizes audio signal processing in multiple modes through mode marking and smooth transition, and improves the smoothness of audio switching and signal coherence.

Description

Technical Field

[0001] The present invention relates to the field of audio processing, and in particular to a multi-mode audio processing method, device, Soc chip and storage medium. Background Art

[0002] With the diversification of audio application scenarios, audio systems need to process single-channel and multi-channel audio signals at the same time. In actual applications, audio devices will switch between single-channel mode and multi-channel mode according to different scenarios, such as switching from a single speaker to a multi-person meeting in a conference system, or switching from a single microphone recording to multi-microphone recording in a recording system. However, this channel mode switching often leads to discontinuity of audio signals, resulting in popping sounds, disconnected sounds, and other problems, affecting the continuity of audio playback and user experience.

[0003] The prior art generally uses a simple fade-in / fade-out or direct switching method when processing channel mode switching. This processing method does not take into account the characteristic differences of audio signals under different channel modes, and is likely to cause distortion and distortion of the audio signal. Summary of the invention

[0004] The main purpose of the present invention is to solve the technical problem that the existing audio processing easily causes distortion and distortion of the audio signal when the processing channel mode is switched;

[0005] A first aspect of the present invention provides a multi-mode audio processing method, the multi-mode audio processing method comprising:

[0006] The input audio signal is framed according to a preset time window length, and effective signal detection and channel quality evaluation are performed on the framed audio signal to obtain a channel feature vector;

[0007] Marking the audio signal after framing according to the channel feature vector to obtain a sequence of audio data frames with the channel mode mark, and judging whether there is a channel mode switch in the audio data frame sequence according to the channel mode mark;

[0008] If so, audio processing is performed on the audio data frame sequence according to the channel feature vector and the channel mode mark, and during the audio processing, a crossover transition time window for channel mode switching is set according to the channel mode mark to obtain a processed data sequence with a transition mark;

[0009] The signal amplitudes between adjacent data frames in the processed data sequence are gradually processed according to the transition mark, and the audio signals before and after the switching are smoothly connected according to the cross-transition time window to obtain a continuous output audio signal.

[0010] Optionally, in a first implementation manner of the first aspect of the present invention, the input audio signal is framed according to a preset time window length, and effective signal detection and channel quality assessment are performed on the framed audio signal to obtain a channel feature vector, which includes:

[0011] Performing frame processing on the input audio signal according to the preset time window length, and performing overlapping window processing on the framed audio signal to obtain an audio data frame and frame information of the audio data frame, wherein the frame information includes frame length information, frame overlapping information and the number of frame sampling points;

[0012] Performing channel detection on the audio data frame to obtain the number of channels, and performing amplitude detection and energy analysis on each channel signal in the audio data frame to obtain frequency domain characteristics of each channel;

[0013] The number of channels, the frequency domain features and the frame information are combined and processed to obtain the channel feature vector.

[0014] Optionally, in a second implementation manner of the first aspect of the present invention, when the channel mode mark is a single-channel mode; performing audio processing on the audio data frame sequence according to the channel feature vector and the channel mode mark includes:

[0015] Detecting and suppressing noise signals in the audio data frame sequence according to the frequency domain features in the channel feature vector;

[0016] Calculating the energy level of the audio data frame sequence after noise reduction, and performing dynamic range control on the signal amplitude of the audio data frame sequence according to the calculation result;

[0017] The equalization parameters of the dynamically adjusted audio data frame sequence are configured based on the frequency domain features in the channel feature vector, and the signal frequency response of the audio data frame sequence is equalized and adjusted according to the equalization parameters.

[0018] Optionally, in a third implementation manner of the first aspect of the present invention, when the channel mode mark is a multi-channel mode; performing audio processing on the audio data frame sequence according to the channel feature vector and the channel mode mark includes:

[0019] Constructing a third-order tensor of the audio data frame sequence according to the number of channels, frequency domain features and frame information in the channel feature vector to obtain a multi-channel audio tensor;

[0020] Calculating the tensor nuclear norm of the multi-channel audio tensor, and building an optimization model based on the calculation result to obtain a convex optimization model of tensor completion;

[0021] Solving the convex optimization model using a convex relaxation technique to obtain a completed multi-channel audio tensor;

[0022] An inverse MDCT transform is performed on the completed multi-channel audio tensor to obtain a processed audio data frame sequence.

[0023] Optionally, in a fourth implementation of the first aspect of the present invention, constructing a third-order tensor for the audio data frame sequence according to the number of channels, frequency domain features, and frame information in the channel feature vector to obtain a multi-channel audio tensor includes:

[0024] Performing MDCT transformation on the audio data frame sequence to obtain a frequency domain data sequence;

[0025] The frequency domain data sequence is grouped according to the number of channels, and each group of frequency domain data is dimensionally divided according to the number of frame sampling points in the frame information to obtain a two-dimensional spectrum matrix of each channel;

[0026] Rearranging the two-dimensional spectrum matrix of each channel based on the frequency domain features to obtain a feature matrix of frequency domain coefficient dimension and frame sample dimension;

[0027] The feature matrices of the channels are combined along the channel dimension according to the number of channels to obtain the multi-channel audio tensor.

[0028] Optionally, in a fifth implementation manner of the first aspect of the present invention, the step of calculating the tensor nuclear norm of the multi-channel audio tensor and constructing an optimization model based on the calculation result to obtain a convex optimization model for tensor completion includes:

[0029] Performing forward slicing processing on the multi-channel audio tensor to obtain a plurality of two-dimensional matrix slices, and performing singular value decomposition on the plurality of two-dimensional matrix slices to obtain tensor singular values;

[0030] Constructing an objective function based on the tensor singular values, and constraining the objective function according to a preset audio data integrity constraint condition to obtain a mathematical model for tensor optimization;

[0031] The mathematical model of the tensor optimization is converted into a convex relaxation form to obtain the convex optimization model of the tensor completion.

[0032] Optionally, in a sixth implementation of the first aspect of the present invention, the step of gradually processing the signal amplitude between adjacent data frames in the processed data sequence according to the transition mark, and smoothly connecting the audio signals before and after the switching according to the cross-transition time window to obtain a continuous output audio signal includes:

[0033] Positioning the data frame with the transition mark in the processed data sequence, and determining the adjacent data frames before and after the switching according to the transition mark, to obtain the data frame pair that needs to be gradually processed;

[0034] Calculating the amplitude weight coefficient of the data frame pair according to the length of the cross transition time window, and performing weighted calculation on the audio signals before and after the switching in the data frame pair according to the amplitude weight coefficient to obtain a smoothly transitioned audio segment;

[0035] The smoothly transitioned audio segment is inserted into a corresponding position in the processed data sequence according to time sequence to obtain the continuous output audio signal.

[0036] A second aspect of the present invention provides a multi-mode audio processing device, the multi-mode audio processing device comprising:

[0037] A frame processing module is used to perform frame processing on the input audio signal according to a preset time window length, and perform effective signal detection and channel quality evaluation on the framed audio signal to obtain a channel feature vector;

[0038] a marking judgment module, configured to mark the channel mode of the framed audio signal according to the channel feature vector, obtain an audio data frame sequence with the channel mode mark, and judge whether the audio data frame sequence has a channel mode switch according to the channel mode mark;

[0039] An audio processing module, configured to perform audio processing on the audio data frame sequence according to the channel feature vector and the channel mode mark if there is a channel mode switch, and during the audio processing, set a crossover transition time window of the channel mode switch according to the channel mode mark to obtain a processed data sequence with a transition mark;

[0040] The output module is used to gradually process the signal amplitude between adjacent data frames in the processed data sequence according to the transition mark, and smoothly connect the audio signals before and after the switching according to the cross transition time window to obtain a continuous output audio signal.

[0041] The third aspect of the present invention provides a Soc chip, comprising: a memory and at least one processor, wherein instructions are stored in the memory, and the memory and the at least one processor are interconnected through lines; the at least one processor calls the instructions in the memory so that the Soc chip executes the steps of the above-mentioned multi-mode audio processing method.

[0042] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the steps of the above-mentioned multi-mode audio processing method.

[0043] The above-mentioned multi-mode audio processing method, device, Soc chip and storage medium, by performing frame processing on the input audio signal, detecting the effective signal and evaluating the channel quality, obtain the channel feature vector; mark the channel mode according to the channel feature vector, generate an audio data frame sequence with the channel mode mark, and determine whether there is a channel mode switch; if so, process the audio data frame sequence according to the channel feature vector and the channel mode mark, set the cross-transition time window of the channel mode switch during the processing, and generate a processed data sequence with a transition mark; perform gradual processing on the signal amplitude of adjacent data frames according to the transition mark, smoothly connect the audio signals before and after the switch within the cross-transition time window, and output a continuous audio signal. The present invention realizes audio signal processing in multiple modes through mode marking and smooth transition, and improves the smoothness of audio switching and signal coherence.

[0044] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0045] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of a first embodiment of a multi-mode audio processing method according to an embodiment of the present invention;

[0047] Figure 2 FIG. 4 is a schematic diagram of an embodiment of a multi-mode audio processing device in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] The terms "including" and "having" and any variations thereof mentioned in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device end including a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or device ends.

[0050] To facilitate understanding of this embodiment, a multi-mode audio processing method disclosed in an embodiment of the present invention is first described in detail. Figure 1 As shown, the method comprises the following steps:

[0051] 101. Performing frame processing on the input audio signal according to a preset time window length, and performing effective signal detection and channel quality evaluation on the framed audio signal to obtain a channel feature vector;

[0052] In one embodiment of the present invention, the step of performing frame processing on the input audio signal according to a preset time window length, and performing effective signal detection and channel quality evaluation on the framed audio signal to obtain a channel feature vector comprises: performing frame processing on the input audio signal according to the preset time window length, and performing overlapping window processing on the framed audio signal to obtain an audio data frame, and frame information of the audio data frame, wherein the frame information comprises frame length information, frame overlapping information and the number of frame sampling points; performing channel detection on the audio data frame to obtain the number of channels, and performing amplitude detection and energy analysis on each channel signal in the audio data frame to obtain frequency domain characteristics of each channel; and combining the channel number, the frequency domain characteristics and the frame information to obtain the channel feature vector.

[0053] Specifically, the processing of the input audio signal first requires frame processing. During the frame processing, the preset time window length is set to 20 milliseconds. The reason for this setting is that the audio signal can be considered to be steady within this time range. After framing, overlapping window processing is required, and the overlapping rate is set to 50%, that is, the adjacent frames overlap by 10 milliseconds. This setting can avoid signal discontinuity caused by framing. When the sampling rate is 48kHz, each frame contains 960 sampling points. The overlapping window processing uses the Hamming window, which can effectively reduce spectral leakage. Through framing and windowing processing, an audio data frame sequence is obtained. At the same time, the frame information such as frame length information (20 milliseconds), frame overlap information (10 milliseconds) and the number of frame sampling points (960 points) are recorded, which will be used for subsequent signal processing and analysis. The results of framing and overlapping window processing directly affect the accuracy of subsequent feature extraction. Therefore, the selection of frame length and the setting of overlap rate need to balance the requirements of time domain resolution and frequency domain resolution. Too short frame length will lead to a decrease in the frequency resolution of frequency domain analysis, and too long frame length will make it difficult to capture the mutation features in the time domain. Overlapping window processing can not only smooth the transition between frames, but also provide additional signal sampling points, increasing the reliability of signal analysis.

[0054] Specifically, the process of channel detection of audio data frames requires analyzing the number of valid signals in each data frame. Channel detection determines the number of channels by identifying independent audio streams in each frame. For each detected channel, amplitude detection and energy analysis are required. Amplitude detection focuses on the dynamic range of the signal and evaluates the signal strength by calculating the maximum, minimum and root mean square value of the signal. Energy analysis calculates the energy distribution of each channel in different frequency bands, and the purpose of this is to obtain the spectral characteristics of the signal. The results of amplitude detection and energy analysis together constitute the frequency domain characteristics of each channel, which reflect the basic characteristics of the signal. In the process of channel detection, the time-varying characteristics of the signal need to be considered, so independent channel analysis must be performed on each frame. Amplitude detection not only considers the amplitude change within a single frame, but also the amplitude relationship between adjacent frames to ensure the stability of the detection results. Energy analysis needs to be performed in multiple frequency bands. Usually, the signal spectrum is divided into multiple sub-bands, and the energy value of each sub-band is calculated separately. This multi-band energy analysis can more comprehensively reflect the frequency domain characteristics of the signal.

[0055] Specifically, the number of channels, frequency domain features and frame information need to be combined into a complete channel feature vector. When combining, the number of channels is first used as the primary parameter of the vector, because the number of channels determines the basic mode of subsequent processing. Then the frequency domain features of each channel are arranged in a specified format, including the amplitude parameters and energy distribution information of each channel. Finally, the frame information is also included in the feature vector, because this information is indispensable in subsequent signal processing. The channel feature vector obtained by this combination contains the complete feature information of the audio signal, which provides the necessary basis for subsequent channel mode judgment and audio processing. The combined feature vector is a structured data, in which each element has a clear physical meaning, which is convenient for subsequent processing programs to parse and use. The composition of the channel feature vector needs to consider the organizational form of the data, so that each component in the vector can accurately reflect the various feature dimensions of the signal. The arrangement order of the vector and the setting of the data format need to be standardized, so as to ensure that the subsequent processing module can correctly interpret these feature information. The feature vector also contains necessary auxiliary information, such as sampling rate, quantization accuracy, etc., which are also necessary for signal reconstruction and processing.

[0056] 102. Mark the audio signal after the frame division into channels in a channel mode according to the channel feature vector to obtain an audio data frame sequence with the channel mode mark, and determine whether the audio data frame sequence has a channel mode switch according to the channel mode mark;

[0057] In one embodiment of the present invention, the mode marking process based on the channel feature vector mainly relies on the channel number information in the channel feature vector. In the channel marking process, it is first necessary to extract the channel number in the channel feature vector and mark each frame in the audio data frame sequence. When the channel number is 1, the frame is marked as a single-channel mode; when the channel number is greater than 1, the frame is marked as a multi-channel mode. The channel mode mark is a binary mark that directly reflects the channel working state of the current audio frame. The marking process needs to traverse the entire audio data frame sequence to ensure that each frame is correctly marked. The marking information will be stored as an additional attribute of the audio frame, and these marking information and the original audio data together constitute an audio data frame sequence with a channel mode mark. The process of judging the channel mode switching requires sequential analysis of the audio data frame sequence with the channel mode mark. The analysis process adopts a sliding window method, and the window length usually contains multiple continuous frames. By comparing the channel mode marks of adjacent frames in the window, when a change in the mark value is detected, it is determined that there is a channel mode switch. For example, when the mark is detected to change from single-channel mode to multi-channel mode, or from multi-channel mode to single-channel mode, it indicates that a channel mode switch has occurred at this location. The judgment of mode switching needs to consider the marking of multiple consecutive frames to avoid misjudgment due to instantaneous interference. In addition to the Boolean value of whether there is a switch, the judgment result also includes the specific location information of the switch, which will be used for subsequent smoothing processing.

[0058] 103. If yes, perform audio processing on the audio data frame sequence according to the channel feature vector and the channel mode mark, and during the audio processing, set a crossover transition time window for channel mode switching according to the channel mode mark to obtain a processed data sequence with a transition mark;

[0059] In one embodiment of the present invention, when audio processing is performed on an audio data frame sequence according to a channel feature vector and a channel mode mark, different processing strategies need to be adopted according to different channel modes. When marked as a single-channel mode, the processing process mainly includes noise detection and suppression, dynamic range control, and equalization adjustment of frequency response, and these processes are parameterized based on the frequency domain features in the channel feature vector. When marked as a multi-channel mode, the audio data needs to be converted into frequency domain information, and a third-order tensor structure is constructed, and the multi-channel audio data is processed by a tensor completion algorithm. This mode-based processing strategy ensures that the audio signal can obtain the most optimized processing effect in different modes. When setting a crossover transition time window during audio processing, it is first necessary to determine the position of the switching point. The crossover transition time window is centered at the switching point, extends forward and backward for several frames, and the window length is usually set to 200-300 milliseconds. Within this window range, special processing needs to be performed on the audio data before and after the switch, including gradual control of the signal amplitude and smooth transition of the spectrum characteristics. At the same time, within the range of the crossover transition time window, a transition mark is added to each frame of data, and the mark contains the relative position information of the frame in the transition window. These transition marks will be used for subsequent smooth connection processing to ensure the continuity of the audio signal when the mode is switched. Through this processing, the final result is a processed data sequence with transition marks, which not only contains the processed audio data, but also contains the necessary control information for smooth switching.

[0060] Further, when the channel mode is marked as a single-channel mode; the audio processing of the audio data frame sequence according to the channel feature vector and the channel mode mark includes: detecting and suppressing the noise signal in the audio data frame sequence according to the frequency domain features in the channel feature vector; calculating the energy level of the denoised audio data frame sequence, and dynamically controlling the signal amplitude of the audio data frame sequence according to the calculation result; configuring the equalization parameters of the dynamically adjusted audio data frame sequence based on the frequency domain features in the channel feature vector, and equalizing the signal frequency response of the audio data frame sequence according to the equalization parameters.

[0061] Specifically, the noise detection and suppression processing of the single-channel audio data frame sequence needs to be performed based on the frequency domain features in the channel feature vector. In the noise detection process, each frame of the audio data frame sequence is first extracted and compared with the frequency domain features in the channel feature vector. The frequency domain features contain the energy distribution information of the signal in each frequency band, and the main frequency components of the signal can be determined by analyzing this information. For each frame of data, the distribution characteristics of the noise signal are obtained by calculating the difference between its spectrum and the frequency domain features. When the energy of a certain frequency band deviates significantly from the normal value in the frequency domain features, the frequency band is marked as a noise band. The noise suppression processing adopts an adaptive filtering method to selectively attenuate the signal components marked as noise bands. The degree of attenuation is dynamically adjusted according to the energy level of the noise band to ensure that the effective signal will not be excessively affected while suppressing the noise. Through this noise detection and suppression processing based on frequency domain features, the noise interference in the single-channel audio signal can be effectively reduced while maintaining the naturalness of the signal.

[0062] Specifically, when dynamic range control is performed on the audio data frame sequence after noise reduction, energy level calculation is first required. The energy level calculation is performed on the basis of frames. For each frame of data, its effective value (RMS value) is calculated as the energy index of the frame. At the same time, the energy changes of multiple frames are statistically analyzed in a longer time window to obtain the dynamic range characteristics of the signal. According to the calculated energy distribution, the threshold of dynamic range control is set, including the compression starting point and the compression ratio. When the signal energy exceeds the compression starting point, the signal amplitude is adjusted according to the set compression ratio. This adjustment is nonlinear, and different compression ratios are used for signals with different energy levels. For signals with too low energy, appropriate gain compensation is performed to ensure the audibility of the signal. The purpose of dynamic range control is to make the energy distribution of the audio signal more balanced and avoid the situation where the signal amplitude is too large or too small. The audio data frame sequence after processing has a more reasonable dynamic range distribution, which is very important for subsequent equalization processing.

[0063] Specifically, when configuring equalization parameters based on the frequency domain features in the channel feature vector, it is necessary to analyze the energy distribution relationship of each frequency band in the frequency domain features. The configuration of the equalization parameters adopts the structure of a multi-band parametric equalizer, and the gain parameters of each frequency band are independently configured according to the frequency domain features. For frequency bands with uneven energy distribution in the frequency domain features, the gain parameters of the corresponding frequency bands are adjusted to compensate, so that the frequency response of the signal is flatter. When configuring the equalization parameters, the auditory characteristics of the human ear must also be considered, and the processing of the mid-frequency band must be particularly cautious because this frequency band is most sensitive to the human ear. The configured equalization parameters are used for subsequent frequency response equalization adjustment. The adjustment process is implemented using a digital filter group, each filter is responsible for the gain control of a frequency band, and the outputs of all filters are superimposed to form the final equalization effect. This equalization processing based on frequency domain features can effectively improve the spectral balance of a single-channel audio signal, making it more natural to listen to. After the equalization processing is completed, the audio signal has a suitable energy distribution in each frequency band, which plays an important role in ensuring the overall sound quality of the audio signal.

[0064] Furthermore, when the channel mode is marked as a multi-channel mode; the audio processing of the audio data frame sequence according to the channel feature vector and the channel mode mark includes: constructing a third-order tensor of the audio data frame sequence according to the number of channels, frequency domain features and frame information in the channel feature vector to obtain a multi-channel audio tensor; calculating the tensor nuclear norm of the multi-channel audio tensor, and constructing an optimization model based on the calculation result to obtain a convex optimization model of tensor completion; solving the convex optimization model using a convex relaxation technique to obtain a completed multi-channel audio tensor; performing an inverse MDCT transform on the completed multi-channel audio tensor to obtain a processed audio data frame sequence.

[0065] Specifically, the process of constructing a third-order tensor for a multi-channel audio data frame sequence requires the comprehensive use of the number of channels, frequency domain features and frame information in the channel feature vector. First, according to the number of sampling points in the frame information, the audio data frame sequence is subjected to MDCT transformation to obtain frequency domain data. The MDCT transformation can convert the time domain signal to the frequency domain. The number of frequency domain coefficients obtained after the transformation is half the number of sampling points per frame, and these frequency domain coefficients constitute the first dimension of the tensor. At the same time, based on the frame length information and frame overlap information in the frame information, the transformed frequency domain data is arranged in the order of the frames to form the second dimension of the tensor. Finally, the frequency domain data of each channel is arranged in the third dimension according to the number of channels, so that a third-order tensor containing the frequency domain coefficient dimension, the frame sample dimension and the channel number dimension is constructed. This three-dimensional structure of the tensor can completely retain all the characteristic information of the multi-channel audio signal, including frequency characteristics, timing relationships and channel correlations.

[0066] Specifically, when calculating the nuclear norm of a multi-channel audio tensor, it is first necessary to perform forward slicing on the tensor to obtain a series of two-dimensional matrix slices. Each slice represents the multi-channel signal characteristics at a specific time or frequency point. These two-dimensional matrix slices are singularly decomposed to obtain the singular values ​​of each slice. The calculation process of the tensor nuclear norm is to sum the singular values ​​of all slices, and this value reflects the rank information of the tensor. Based on the calculated tensor nuclear norm, the objective function of the optimization model is constructed. The objective function usually consists of two parts: one part is the tensor nuclear norm term, which is used to control the rank of the reconstructed signal; the other part is the data term, which is used to ensure the consistency of the reconstructed signal with the known data. At the same time, according to the integrity requirements of the audio data, the corresponding constraints are added, and finally a complete tensor completion convex optimization model is obtained.

[0067] Specifically, the process of solving the optimization model using convex relaxation technology is carried out in an iterative manner. First, the original non-convex optimization problem is transformed into a convex optimization problem. This transformation is achieved by appropriately relaxing and approximating the objective function. In each iteration, some variables are fixed first, the optimal values ​​of the other part of the variables are solved, and then the other variables are optimized alternately. In this process, the Lagrange multiplier method is used to handle the constraints and construct the augmented Lagrangian function. The solution is solved by the alternating direction multiplier method (ADMM), and each iteration includes three steps: original variable update, dual variable update, and multiplier update. The iterative process will continue until the preset convergence condition is met or the maximum number of iterations is reached. The final completed multi-channel audio tensor maintains the structural characteristics of the original signal while supplementing the missing data information.

[0068] Specifically, the process of inverse MDCT transforming the completed multi-channel audio tensor is to reconstruct the frequency domain data into a time domain signal. First, the frequency domain data of each channel is extracted from the third-order tensor, and these data are arranged in the order of frames to form multiple frequency domain matrices. These frequency domain matrices are respectively subjected to inverse MDCT transform to obtain the time domain signals of each channel. The influence of frame overlap needs to be considered during the transformation process, and the frame boundary effect is eliminated through reasonable overlap-add processing. After the time domain signal obtained by the inverse transform is overlap-add processed, a continuous audio data stream is formed. The audio data of each channel is reorganized according to the original channel order, and finally a processed multi-channel audio data frame sequence is obtained. This data frame sequence maintains the channel characteristics and timing relationship of the original signal, and at the same time improves the signal quality through tensor completion processing. The overlap-add technology used in the inverse transform process ensures the continuity and smoothness of the reconstructed signal and avoids audio distortion caused by frame processing.

[0069] Furthermore, the third-order tensor construction of the audio data frame sequence according to the number of channels, frequency domain features and frame information in the channel feature vector to obtain a multi-channel audio tensor includes: performing MDCT transform on the audio data frame sequence to obtain a frequency domain data sequence; grouping the frequency domain data sequence into channels according to the number of channels, and dividing each group of frequency domain data into dimensions according to the number of frame sampling points in the frame information to obtain a two-dimensional spectrum matrix of each channel; rearranging the two-dimensional spectrum matrix of each channel based on the frequency domain features to obtain a feature matrix of frequency domain coefficient dimension and frame sample dimension; and combining the feature matrices of each channel along the channel dimension according to the number of channels to obtain the multi-channel audio tensor.

[0070] Specifically, the process of performing MDCT transformation on the audio data frame sequence first needs to ensure that each frame of data meets the requirements of MDCT transformation. According to the parameters in the frame information, the audio frame length is 20ms, the frame overlap is 10ms, and when the sampling rate is 48kHz, each frame contains 960 sampling points. MDCT transformation is an improved discrete cosine transform, which is characterized by 50% overlap between adjacent frames, which matches the overlap rate of audio frames. Before MDCT transformation, each frame signal needs to be windowed, and a suitable window function such as Hamming window is used to reduce spectrum leakage. The transformation process converts the 960-point time domain data of each frame into 480-point frequency domain coefficients, which reflect the energy distribution of the signal at different frequencies. After MDCT transformation of all frames, a complete frequency domain data sequence is obtained, which retains the time-frequency characteristics of the signal, and can achieve perfect reconstruction due to the symmetric characteristics of MDCT.

[0071] Specifically, the process of grouping and dimensioning the frequency domain data sequence according to the number of channels needs to consider the organization form of the data. When grouping channels, the frequency domain data sequence is classified according to the channel identifier according to the channel number information in the channel feature vector, and the data of each channel constitutes an independent data group. For the frequency domain data of each channel group, it is necessary to dimensionally divide it according to the number of frame sampling points in the frame information. The dimension division process is actually to reorganize the linear frequency domain data sequence into a matrix form, where the rows of the matrix correspond to the frequency domain coefficients (480 points) and the columns correspond to different time frames. This two-dimensional arrangement forms a frequency-time representation, which is called a two-dimensional spectrum matrix. The two-dimensional spectrum matrix of each channel has the same dimensional structure, where the size of the frequency domain dimension is determined by the number of coefficients after the MDCT transform, and the size of the time dimension is determined by the number of frames. Through this organization method, the time-frequency characteristics of each channel are clearly expressed in the corresponding two-dimensional spectrum matrix.

[0072] Specifically, when rearranging the two-dimensional spectrum matrix of each channel based on the frequency domain features in the channel feature vector, it is necessary to refer to the spectrum distribution information recorded in the frequency domain features. The purpose of rearrangement is to make the structure of the spectrum matrix more suitable for subsequent tensor operations while maintaining the frequency domain characteristics of the signal. The rearrangement process must first determine the arrangement order of the frequency domain coefficient dimension and the frame sample dimension, and this order must take into account the local correlation of the data. In the frequency domain dimension, adjacent frequency points often have a strong correlation, so the continuity of the frequency points should be maintained as much as possible. In the frame sample dimension, the spectrum changes between adjacent frames are usually smooth, and this temporal continuity also needs to be maintained during rearrangement. The feature matrix obtained after rearrangement is still a two-dimensional structure, but the arrangement of the elements therein is more conducive to subsequent tensor processing.

[0073] Specifically, the process of combining the feature matrices of each channel along the channel dimension is actually the last step in constructing a third-order tensor. The feature matrices of all channels are superimposed in a fixed order according to the number of channels to form a three-dimensional data structure. In this structure, the first dimension is the frequency domain coefficient, which reflects the frequency characteristics of the signal; the second dimension is the frame sample, which represents the time series information; the third dimension is the number of channels, which contains spatial distribution information. Such a three-dimensional structure completely retains all the characteristics of the multi-channel audio signal, including information in three aspects: time, frequency and space. The obtained multi-channel audio tensor is a regular data structure, and each element has a clear physical meaning: its specific position in frequency, time and space. This structured data organization form facilitates subsequent tensor operations, and also ensures the integrity of various feature information in the signal processing process.

[0074] Furthermore, the tensor nuclear norm of the multi-channel audio tensor is calculated, and an optimization model is constructed based on the calculation result to obtain a convex optimization model for tensor completion, including: forward slicing the multi-channel audio tensor to obtain multiple two-dimensional matrix slices, and singular value decomposition of the multiple two-dimensional matrix slices to obtain tensor singular values; constructing an objective function based on the tensor singular values, and constraining the objective function according to preset audio data integrity constraints to obtain a mathematical model for tensor optimization; converting the mathematical model for tensor optimization into a convex relaxation form to obtain the convex optimization model for tensor completion.

[0075] Specifically, when forward slicing a multi-channel audio tensor, the third-order tensor needs to be sliced ​​according to the channel dimension. Forward slicing refers to obtaining slices of a two-dimensional plane while fixing the third dimension (channel dimension). Each slice corresponds to a two-dimensional matrix of a channel in the frequency domain coefficient dimension and the frame sample dimension. For each two-dimensional matrix slice, a singular value decomposition operation needs to be performed. Singular value decomposition decomposes the matrix into the product of three matrices: a left singular vector matrix, a singular value diagonal matrix, and a right singular vector matrix. In this decomposition process, the most critical thing is to obtain singular values, which are arranged in the diagonal matrix from large to small. The singular values ​​of all slices together constitute the singular values ​​of the tensor, which reflect the energy distribution characteristics of the tensor in different dimensions. The combination of forward slicing and singular value decomposition transforms the originally complex third-order tensor problem into multiple two-dimensional matrix problems. This transformation greatly simplifies the subsequent calculation process. At the same time, singular value decomposition provides a low-rank approximation basis for tensors, which is of great significance for building optimization models.

[0076] Specifically, when constructing the objective function based on tensor singular values, the first task is to establish the tensor nuclear norm expression. The tensor nuclear norm is defined as the sum of the singular values ​​of all slices. This definition is a natural extension of the matrix nuclear norm on high-order tensors. The objective function consists of two parts: the tensor nuclear norm term and the data fitting term. The tensor nuclear norm term is used to control the rank of the reconstructed signal, while the data fitting term ensures the consistency between the reconstructed signal and the known data. When constructing the objective function, a weight parameter needs to be introduced to balance the relative importance of these two terms. At the same time, according to the characteristics of audio data, integrity constraints need to be added. These constraints include: the values ​​of known data points must remain unchanged, the amplitude of the reconstructed data should be within a reasonable range, and the changes between adjacent frames should be continuous. These constraints together ensure the physical rationality of the reconstruction results. The mathematical model constructed in this way contains both the optimization objective and the necessary constraints, forming a complete optimization problem.

[0077] Specifically, the mathematical model of tensor optimization is converted into a convex relaxation form to facilitate the solution. The original tensor completion problem is usually non-convex and difficult to solve directly. Through the convex relaxation technology, the non-convex optimization problem is approximated as a convex optimization problem. This conversion process mainly includes two aspects: one is to replace the non-convex objective function with a convex objective function, and the other is to replace the non-convex constraint set with a convex constraint set. In terms of the objective function, the tensor nuclear norm is used as the convex envelope of the rank function, which is a good convex relaxation. In terms of constraints, some nonlinear constraints are converted into convex constraints by linearization or quadratic methods. The convex optimization model obtained after the conversion has good mathematical properties, which not only guarantees the existence of the global optimal solution, but also can be solved by a standard convex optimization algorithm. This conversion greatly simplifies the solution process and makes the tensor completion problem feasible in practical applications.

[0078] 104. Performing gradual processing on the signal amplitudes between adjacent data frames in the processed data sequence according to the transition mark, and smoothly connecting the audio signals before and after the switching according to the cross transition time window to obtain a continuous output audio signal.

[0079] In one embodiment of the present invention, the step of gradually processing the signal amplitude between adjacent data frames in the processed data sequence according to the transition mark, and smoothly connecting the audio signals before and after the switching according to the cross-transition time window to obtain a continuous output audio signal includes: locating the data frames with transition marks in the processed data sequence, and determining the adjacent data frames before and after the switching according to the transition mark to obtain a pair of data frames that need to be gradually processed; calculating the amplitude weight coefficient of the data frame pair according to the length of the cross-transition time window, and performing weighted calculation on the audio signals before and after the switching in the data frame pair according to the amplitude weight coefficient to obtain a smoothly transitioned audio segment; and inserting the smoothly transitioned audio segment into a corresponding position in the processed data sequence according to the time sequence to obtain the continuous output audio signal.

[0080] Specifically, when locating a data frame with a transition mark in a processing data sequence, it is first necessary to scan the entire processing data sequence to find the position of the data frame containing the transition mark. The transition mark indicates the moment when the channel mode is switched. The purpose of locating these data frames is to determine the specific area that needs to be smoothed. When the transition mark is found, it is necessary to determine the adjacent data frames before and after the marker position. In the forward search process, the last few frames of data before the switch are determined; in the backward search process, the starting few frames of data after the switch are determined. These adjacent data frames constitute the data frame pairs that need to be gradually processed. The selection of data frame pairs needs to consider the continuity of the frames to ensure that there is no data loss or overlap between the selected frames. At the same time, it is also necessary to record the position information of these data frame pairs in the original sequence, which will be used for the subsequent smooth audio segment insertion. Each data frame pair contains the audio state information before and after the switch, which is crucial for generating a smooth transition effect. The selection range of the frame pair needs to match the length of the cross transition time window to ensure that there is enough data for gradual processing.

[0081] Specifically, calculating the amplitude weight coefficient of the data frame pair according to the length of the cross-transition time window is the core step of the gradual change process. The cross-transition time window is usually set to 200-300 milliseconds. This length needs to be long enough to produce a smooth transition effect, but short enough to avoid a noticeable sense of delay. When calculating the weight coefficient, a nonlinear weight distribution function is used so that the weight value changes faster near the center of the window and changes slower at both ends of the window. This nonlinear distribution can produce a more natural transition effect. The calculation of the weight coefficient needs to take into account the dynamic characteristics of the audio signal. For areas where the signal amplitude changes greatly, the rate of change of the weight coefficient needs to be adjusted accordingly. The calculated weight coefficient is used to perform weighted superposition on the audio signals before and after the switch. Through this weighted calculation, the signal before the switch gradually fades out, and the signal after the switch gradually fades in, thereby forming a smooth transition. When performing the weighted calculation, it is necessary to ensure that the sum of the weights at each moment is 1, so as to avoid sudden changes in signal energy. The resulting audio clip is continuous in time and smoothly transitioned in amplitude.

[0082] Specifically, the process of inserting a smoothly transitioned audio clip into a processed data sequence requires precise control of the insertion position and timing relationship. The insertion position is determined by the position information of the previously recorded data frame pair, and the smoothly transitioned audio clip replaces the original switching area audio data. During the insertion process, it is necessary to ensure that the timestamp of the audio clip is aligned with the original sequence to avoid timing confusion caused by the insertion operation. The inserted audio data needs to be connected with the previous and next clips to ensure that there will be no jumps or discontinuities at the clip connection. After the insertion operation is completed, the entire audio sequence is continuous in time and changes smoothly in amplitude. This smoothing process not only eliminates the sense of mutation when the channel mode is switched, but also maintains the naturalness of the audio signal. The final output continuous audio signal has no obvious switching traces in the sense of hearing, and achieves seamless channel mode conversion. At the same time, this processing method also ensures that the dynamic range and spectral characteristics of the signal will not change drastically due to mode switching.

[0083] In this embodiment, the input audio signal is processed by framing, valid signals are detected and channel quality is evaluated to obtain a channel feature vector; the channel mode is marked according to the channel feature vector, an audio data frame sequence with the channel mode mark is generated, and it is determined whether there is a channel mode switch; if so, the audio data frame sequence is processed according to the channel feature vector and the channel mode mark, and a cross-transition time window for channel mode switching is set during the processing to generate a processed data sequence with a transition mark; the signal amplitudes of adjacent data frames are gradually processed according to the transition mark, and the audio signals before and after the switch are smoothly connected within the cross-transition time window, and a continuous audio signal is output. The present invention realizes audio signal processing in multiple modes through mode marking and smooth transition, and improves the smoothness of audio switching and signal coherence.

[0084] The above describes the multi-mode audio processing method in the embodiment of the present invention. The following describes the multi-mode audio processing device in the embodiment of the present invention. Figure 2 , an embodiment of the multi-mode audio processing device in the embodiment of the present invention includes:

[0085] The frame processing module 201 is used to perform frame processing on the input audio signal according to a preset time window length, and perform effective signal detection and channel quality evaluation on the framed audio signal to obtain a channel feature vector;

[0086] The marking judgment module 202 is used to mark the channel mode of the framed audio signal according to the channel feature vector to obtain an audio data frame sequence with the channel mode mark, and judge whether the audio data frame sequence has a channel mode switch according to the channel mode mark;

[0087] The audio processing module 203 is used for performing audio processing on the audio data frame sequence according to the channel feature vector and the channel mode mark if there is a channel mode switch, and in the audio processing process, setting a crossover transition time window of the channel mode switch according to the channel mode mark to obtain a processed data sequence with a transition mark;

[0088] The output module 204 is used to perform gradual processing on the signal amplitude between adjacent data frames in the processed data sequence according to the transition mark, and smoothly connect the audio signals before and after the switching according to the cross transition time window to obtain a continuous output audio signal.

[0089] In an embodiment of the present invention, the multi-mode audio processing device runs the above-mentioned multi-mode audio processing method, and the multi-mode audio processing device performs frame processing on the input audio signal, detects the effective signal and evaluates the channel quality to obtain the channel feature vector; performs channel mode marking according to the channel feature vector, generates an audio data frame sequence with the channel mode mark, and determines whether there is a channel mode switch; if so, processes the audio data frame sequence according to the channel feature vector and the channel mode mark, sets the cross-transition time window of the channel mode switch during the processing, and generates a processed data sequence with a transition mark; performs gradual processing on the signal amplitude of adjacent data frames according to the transition mark, smoothly connects the audio signals before and after the switch within the cross-transition time window, and outputs a continuous audio signal. The present invention realizes audio signal processing in multiple modes through mode marking and smooth transition, and improves the smoothness of audio switching and signal coherence.

[0090] above Figure 2 The multi-mode audio processing device in the embodiment of the present invention is described in detail from the perspective of modular functional entities, and the Soc chip in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0091] The Soc chip may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) (for example, one or more processors) and memories, and one or more storage media for storing applications or data (for example, one or more mass storage devices). Among them, the memory and the storage medium can be short-term storage or permanent storage. The program stored in the storage medium 3 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the Soc chip. Furthermore, the processor can be configured to communicate with the storage medium and execute a series of instruction operations in the storage medium on the Soc chip to implement the steps of the above-mentioned multi-mode audio processing method.

[0092] The Soc chip may also have one or more wired or wireless network interfaces, one or more input and output interfaces, and / or one or more operating systems, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will appreciate that the Soc chip structure does not limit the Soc chip provided by the present invention, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0093] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the multi-mode audio processing method.

[0094] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0095] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0096] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-mode audio processing method, characterized in that: The multi-mode audio processing method comprises: The input audio signal is framed according to a preset time window length, and effective signal detection and channel quality evaluation are performed on the audio signal after framing to obtain a channel feature vector; the step of framing the input audio signal according to the preset time window length, and performing effective signal detection and channel quality evaluation on the audio signal after framing to obtain a channel feature vector comprises: framing the input audio signal according to the preset time window length, and performing overlapping window processing on the audio signal after framing to obtain an audio data frame, and frame information of the audio data frame, wherein the frame information comprises frame length information, frame overlapping information and the number of frame sampling points; performing channel detection on the audio data frame to obtain the number of channels, and performing amplitude detection and energy analysis on each channel signal in the audio data frame to obtain the frequency domain characteristics of each channel; combining the number of channels, the frequency domain characteristics and the frame information to obtain the channel feature vector; Marking the audio signal after framing according to the channel feature vector to obtain a sequence of audio data frames with the channel mode mark, and judging whether there is a channel mode switch in the audio data frame sequence according to the channel mode mark; If so, audio processing is performed on the audio data frame sequence according to the channel feature vector and the channel mode mark, and during the audio processing, a crossover transition time window for channel mode switching is set according to the channel mode mark to obtain a processed data sequence with a transition mark; The signal amplitudes between adjacent data frames in the processed data sequence are gradually processed according to the transition mark, and the audio signals before and after the switching are smoothly connected according to the cross-transition time window to obtain a continuous output audio signal.

2. The multi-mode audio processing method according to claim 1, characterized in that: When the channel mode mark is a single channel mode; performing audio processing on the audio data frame sequence according to the channel feature vector and the channel mode mark comprises: Detecting and suppressing noise signals in the audio data frame sequence according to the frequency domain features in the channel feature vector; Calculating the energy level of the audio data frame sequence after suppressing the noise signal, and controlling the dynamic range of the signal amplitude of the audio data frame sequence according to the calculation result; The equalization parameters of the audio data frame sequence after dynamic range control are configured based on the frequency domain characteristics in the channel feature vector, and the signal frequency response of the audio data frame sequence is equalized and adjusted according to the equalization parameters.

3. The multi-mode audio processing method according to claim 1, characterized in that: When the channel mode mark is a multi-channel mode; performing audio processing on the audio data frame sequence according to the channel feature vector and the channel mode mark includes: Constructing a third-order tensor of the audio data frame sequence according to the number of channels, frequency domain features and frame information in the channel feature vector to obtain a multi-channel audio tensor; Calculating the tensor nuclear norm of the multi-channel audio tensor, and building an optimization model based on the calculation result to obtain a convex optimization model of tensor completion; Solving the convex optimization model using a convex relaxation technique to obtain a completed multi-channel audio tensor; An inverse MDCT transform is performed on the completed multi-channel audio tensor to obtain a processed audio data frame sequence.

4. The multi-mode audio processing method according to claim 3, characterized in that: The constructing of a third-order tensor of the audio data frame sequence according to the number of channels, frequency domain features and frame information in the channel feature vector to obtain a multi-channel audio tensor includes: Performing MDCT transformation on the audio data frame sequence to obtain a frequency domain data sequence; The frequency domain data sequence is grouped according to the number of channels, and each group of frequency domain data is dimensionally divided according to the number of frame sampling points in the frame information to obtain a two-dimensional spectrum matrix of each channel; Rearranging the two-dimensional spectrum matrix of each channel based on the frequency domain features to obtain a feature matrix of frequency domain coefficient dimension and frame sample dimension; The feature matrices of the channels are combined along the channel dimension according to the number of channels to obtain the multi-channel audio tensor.

5. The multi-mode audio processing method according to claim 3, characterized in that: The step of calculating the tensor nuclear norm of the multi-channel audio tensor and constructing an optimization model based on the calculation result to obtain a convex optimization model for tensor completion includes: Performing forward slicing processing on the multi-channel audio tensor to obtain a plurality of two-dimensional matrix slices, and performing singular value decomposition on the plurality of two-dimensional matrix slices to obtain tensor singular values; Constructing an objective function based on the tensor singular values, and constraining the objective function according to a preset audio data integrity constraint condition to obtain a mathematical model for tensor optimization; The mathematical model of the tensor optimization is converted into a convex relaxation form to obtain the convex optimization model of the tensor completion.

6. The multi-mode audio processing method according to claim 1, characterized in that: The step of gradually processing the signal amplitude between adjacent data frames in the processed data sequence according to the transition mark, and smoothly connecting the audio signals before and after the switching according to the cross-transition time window to obtain a continuous output audio signal comprises: Positioning the data frame with the transition mark in the processed data sequence, and determining the adjacent data frames before and after the switching according to the transition mark, to obtain the data frame pair that needs to be gradually processed; Calculating the amplitude weight coefficient of the data frame pair according to the length of the cross transition time window, and performing weighted calculation on the audio signals before and after the switching in the data frame pair according to the amplitude weight coefficient to obtain a smoothly transitioned audio segment; The smoothly transitioned audio segment is inserted into a corresponding position in the processed data sequence according to time sequence to obtain the continuous output audio signal.

7. A multi-mode audio processing device, characterized in that: The multi-mode audio processing device comprises: The frame processing module is used to perform frame processing on the input audio signal according to a preset time window length, and perform effective signal detection and channel quality evaluation on the audio signal after framing to obtain a channel feature vector; the frame processing on the input audio signal according to the preset time window length, and perform effective signal detection and channel quality evaluation on the audio signal after framing to obtain a channel feature vector includes: performing frame processing on the input audio signal according to the preset time window length, and performing overlapping window processing on the audio signal after framing to obtain an audio data frame, and frame information of the audio data frame, wherein the frame information includes frame length information, frame overlapping information and the number of frame sampling points; performing channel detection on the audio data frame to obtain the number of channels, and performing amplitude detection and energy analysis on each channel signal in the audio data frame to obtain the frequency domain characteristics of each channel; combining the number of channels, the frequency domain characteristics and the frame information to obtain the channel feature vector; a marking judgment module, configured to mark the channel mode of the framed audio signal according to the channel feature vector, obtain an audio data frame sequence with the channel mode mark, and judge whether the audio data frame sequence has a channel mode switch according to the channel mode mark; An audio processing module, configured to perform audio processing on the audio data frame sequence according to the channel feature vector and the channel mode mark if there is a channel mode switch, and during the audio processing, set a crossover transition time window of the channel mode switch according to the channel mode mark to obtain a processed data sequence with a transition mark; The output module is used to gradually process the signal amplitude between adjacent data frames in the processed data sequence according to the transition mark, and smoothly connect the audio signals before and after the switching according to the cross transition time window to obtain a continuous output audio signal.

8. A Soc chip, characterized in that: The Soc chip includes: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory so that the Soc chip executes the steps of the multi-mode audio processing method as described in any one of claims 1-6.

9. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by a processor, the steps of the multi-mode audio processing method as described in any one of claims 1 to 6 are implemented.

Images