Audio decoding performance optimization method and device for multi-mode SoC master chip

Abstract

The present application relates to the field of audio decoding technology, and discloses a method and device for optimizing the audio decoding performance of a multi-mode SoC main control chip, the method comprising: performing frame processing on the input audio data through a DSP processing unit, and extracting audio feature data; performing period detection and multi-frame smoothing processing on the audio feature data to obtain an audio period feature vector; performing principal component transformation and feature dimension reduction based on the audio period feature vector to obtain decoding prediction data, and inputting the decoding prediction data into a task partitioning model for decoding processing sequence analysis to obtain decoding task data; performing hidden Markov modeling on the decoding task data, and calculating and obtaining decoding optimization data; mapping the decoding optimization data to the hardware acceleration unit of the multi-mode SoC main control chip, performing parallel decoding operations, and obtaining target decoding data. The present application realizes adaptive optimization of the decoding process, and can flexibly adjust the decoding strategy according to different audio data characteristics and processing requirements.

Description

Technical Field

[0001] The present application relates to the field of audio decoding technology, and in particular to a method and device for optimizing the audio decoding performance of a multi-mode SoC master control chip. Background Art

[0002] Multi-mode SoC master chips are widely used in the field of audio processing due to their high integration and rich functions. However, traditional audio decoding methods often face problems such as low decoding efficiency and high resource consumption when processing complex audio data.

[0003] In practical applications, the audio decoding process of the multi-mode SoC main control chip involves a large amount of data processing and computing tasks. Due to the lack of effective task division and parallel processing mechanism, the processor load is unevenly distributed, the decoding delay increases, and it also causes a large power loss. Especially when processing high-bitrate audio data, the bottleneck of decoding performance is more obvious. Summary of the invention

[0004] The present application provides a method and device for optimizing the audio decoding performance of a multi-mode SoC master chip. The present application realizes adaptive optimization of the decoding process and can flexibly adjust the decoding strategy according to different audio data characteristics and processing requirements.

[0005] The first aspect of the present application provides a method for optimizing the audio decoding performance of a multi-mode SoC main control chip, and the method for optimizing the audio decoding performance of the multi-mode SoC main control chip includes:

[0006] The DSP processing unit performs frame processing on the input audio data and extracts audio feature data;

[0007] Performing period detection and multi-frame smoothing processing on the audio feature data to obtain an audio period feature vector;

[0008] Performing principal component transformation and feature dimension reduction based on the audio period feature vector to obtain decoding prediction data, and inputting the decoding prediction data into a preset task division model to perform decoding processing sequence analysis to obtain decoding task data;

[0009] Performing hidden Markov modeling on the decoding task data, and obtaining decoding optimization data by calculating through an improved Viterbi algorithm;

[0010] The decoding optimization data is mapped to the hardware acceleration unit of the multi-mode SoC main control chip, and parallel decoding operations are performed to obtain target decoding data.

[0011] The second aspect of the present application provides an audio decoding performance optimization device for a multi-mode SoC main control chip, the audio decoding performance optimization device for the multi-mode SoC main control chip comprising:

[0012] The frame processing module is used to perform frame processing on the input audio data through the DSP processing unit and extract audio feature data;

[0013] A period detection module, used for performing period detection and multi-frame smoothing processing on the audio feature data to obtain an audio period feature vector;

[0014] A sequence analysis module, used for performing principal component transformation and feature dimension reduction based on the audio period feature vector to obtain decoding prediction data, and inputting the decoding prediction data into a preset task division model to perform decoding processing sequence analysis to obtain decoding task data;

[0015] A modeling module, used for performing hidden Markov modeling on the decoding task data, and obtaining decoding optimization data by calculating through an improved Viterbi algorithm;

[0016] The decoding operation module is used to map the decoding optimization data to the hardware acceleration unit of the multi-mode SoC main control chip, perform parallel decoding operations, and obtain target decoding data.

[0017] The third aspect of the present application provides an electronic device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the electronic device executes the above-mentioned audio decoding performance optimization method of the multi-mode SoC master control chip.

[0018] A fourth aspect of the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, the computer executes the above-mentioned method for optimizing the audio decoding performance of the multi-mode SoC master control chip.

[0019] Compared with the prior art, the present application has the following beneficial effects: by extracting audio feature data through the DSP processing unit and combining the multi-frame smoothing processing mechanism, the anti-noise ability of the decoding process is improved, making the feature extraction more accurate and reliable. The principal component analysis method is used to reduce the dimension of the audio features, which effectively reduces data redundancy, reduces the computational complexity, and improves the efficiency of feature extraction. The improved hidden Markov model and Viterbi algorithm are introduced, and the prediction accuracy of the state sequence is optimized by combining the audio cycle feature vector as the weight coefficient. The dynamic task partitioning model and dependency graph are constructed to achieve the reasonable allocation of decoding tasks and improve the utilization rate of system resources. The data pipeline mechanism and dynamic clock control strategy are adopted to achieve efficient parallel decoding processing. A complete performance evaluation and dynamic adjustment mechanism is established, and the system energy consumption is reduced through real-time monitoring and optimization strategies. The hardware acceleration unit and the intelligent power consumption management module are integrated to significantly improve the overall performance of the system. The adaptive optimization of the decoding process is realized, and the decoding strategy can be flexibly adjusted according to different audio data characteristics and processing requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0021] The structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with this technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantive technical significance. Any structural modification, change in proportion or adjustment of size, without affecting the effects and purposes that can be achieved by the present invention, should still fall within the scope of the technical contents disclosed by the present invention.

[0022] Figure 1 It is a flow chart of a method for optimizing the audio decoding performance of a multi-mode SoC master control chip provided by an embodiment of the present invention;

[0023] Figure 2 It is a schematic block diagram of the structure of an audio decoding performance optimization device for a multi-mode SoC main control chip provided by an embodiment of the present invention;

[0024] Figure 3 It is a schematic block diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0027] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0028] It should be further understood that the term "and / or" used in the specification and appended claims of this application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations. Figure 1 , an embodiment of the method for optimizing the audio decoding performance of the multi-mode SoC master chip in the embodiment of the present application includes:

[0029] Step 100: Frame the input audio data through the DSP processing unit and extract audio feature data;

[0030] It is understandable that the execution subject of the present application can be an audio decoding performance optimization device for a multi-mode SoC main control chip, or a terminal or a server, which is not limited here. The present application embodiment is described by taking a server as the execution subject as an example.

[0031] Specifically, the input audio data is segmented by the DSP processing unit, and the continuous audio signal is divided into multiple audio data segments according to the time axis. Discrete sampling operation is performed on each audio data segment according to a fixed number of sampling points, and equally spaced sampling points are extracted from the analog signal to obtain a discrete signal representation. In order to adapt to the needs of different audio processing tasks, a resampling operation is performed on the discrete signal, and the number of sampling points is adjusted to a unified standard to generate framed audio data. In the DSP processing unit, the sampling points in each frame of framed audio data are squared, and all square values ​​are accumulated. The accumulated result reflects the total energy of the current frame audio signal. The accumulated result is divided by the number of sampling points of the corresponding frame to generate energy data, reflecting the overall strength of the audio signal. At the same time, in the analysis of the amplitude change of the audio signal, the positive and negative changes of the amplitude are detected by performing sign bit judgment on the amplitude of adjacent sampling points in each frame of framed audio data, and the total number of sign changes in the entire frame is counted. This statistical value is defined as the number of zero crossings. By dividing it by the total length of the frame, zero crossing data is generated, which reflects the frequency characteristics of the signal in the frame and the rough information of the spectrum energy distribution, especially for pulsed audio or periodic oscillation signals. In order to extract the timing and periodic characteristics of the audio signal, the autocorrelation analysis method is used to process the framed audio data of each frame. According to the preset maximum number of delay points, the original signal is multiplied and accumulated with its delayed signal, and the initial autocorrelation values ​​corresponding to different delay points are calculated. These initial autocorrelation values ​​reflect the periodic characteristics of the signal by capturing the similarity of the signal under different delay conditions. The calculated value of each delay point is divided by the autocorrelation value of the zero delay point to obtain normalized autocorrelation data, eliminating the interference of amplitude on feature extraction. A feature mapping relationship is established based on energy data, zero crossing data and autocorrelation data to obtain audio feature data.

[0032] Step 200, performing period detection and multi-frame smoothing processing on the audio feature data to obtain an audio period feature vector;

[0033] Specifically, the autocorrelation data in the audio feature data is input into the local extreme value detection unit, and the change trend of the signal is analyzed one by one by calculating the amplitude difference of adjacent data points. On the basis of the differential calculation, the position coordinates of the peaks and troughs in the signal are identified, and the positions of these extreme points are used to determine the periodic point sequence, reflecting the periodic characteristics of the audio signal on the time axis. The time coordinate difference operation is performed on the adjacent peak positions in the periodic point sequence, and the time interval between each pair of adjacent peaks is calculated to obtain the difference operation result. The time interval data is converted into fundamental frequency domain data, that is, fundamental frequency sequence data, through a preset formula. This process depends on the physical properties of the signal, and the time interval is reflected as the frequency characteristics of the signal. The generated fundamental frequency sequence can characterize the fundamental frequency distribution characteristics of the signal. The fundamental frequency sequence data is grouped and the statistical characteristics are calculated, and the core characteristics of the fundamental frequency distribution, such as the mean, variance, mode, etc., are extracted by using statistical analysis methods. These statistical features effectively summarize the main laws and change trends of the fundamental frequency sequence. Since there are some missing or abnormal values ​​in the actual audio data, in order to improve the integrity and reliability of the data, linear interpolation is performed on the fundamental frequency statistical data, and the linear relationship between adjacent data is used to supplement the missing values ​​to generate smooth fundamental frequency data to ensure the continuity and accuracy of the fundamental frequency features. At the same time, the energy data in the audio feature data is aligned with the smoothed fundamental frequency data according to the time index, and the fundamental frequency points with energy values ​​less than the preset energy threshold are found during the alignment process. Energy attenuation operation is performed at these low-energy data points, that is, by reducing the fundamental frequency weight values ​​of these points, the impact of noise or meaningless data is minimized. The zero-crossing data in the audio feature data is also aligned with the energy-corrected data according to the time index, and weighted suppression is performed for the data points whose corresponding zero-crossing rates in the energy correction data exceed the preset range. The weighted suppression method is implemented by reducing the weight of the abnormal zero-crossing rate points, thereby reducing the decoding error caused by the violent signal fluctuations. The stability and accuracy of the feature data are guaranteed by correction measures. After completing the energy correction and zero-crossing correction, all feature correction data are input into the multi-frame joint filtering unit, and the continuous multi-frame data is recursively filtered and calculated by the sliding window technology. Each time the sliding window moves, it combines the data of the current frame and its adjacent frames for comprehensive calculation to achieve time-series smoothing of the feature data. This process can effectively suppress the interference of short-term random fluctuations on decoding performance, making the processed feature data smoother and more stable. Based on the time-series smoothed data generated above, a periodic feature index structure is constructed. This structure integrates the fundamental frequency information, energy weight and zero-crossing mark, and combines this information into an audio period feature vector through a mapping algorithm.

[0034] Step 300: Perform principal component transformation and feature dimension reduction based on the audio period feature vector to obtain decoding prediction data, and input the decoding prediction data into a preset task division model to perform decoding processing sequence analysis to obtain decoding task data;

[0035] It should be noted that the state number threshold and time window length are set for the audio cycle feature vector so that the data of the audio cycle feature vector can be divided into discrete states within a fixed time window. Through the quantitative classification algorithm, the feature vector is mapped to a preset discrete state set according to its attributes, and state sequence data is generated to describe the dynamic change law of the audio feature within the time window. The state sequence data is input into the state count matrix to record the number of transitions between adjacent states. In the state count matrix, each element corresponds to the number of occurrences of transitions from one state to another, and the conversion frequency between states is intuitively reflected by counting these transition times. In order to make the transition probability have a standardized meaning, the probability normalization operation is performed on each element of the state count matrix to obtain the probability initial matrix, which provides the initial probability distribution of the transition between states. On the basis of the probability initial matrix, the time series attenuation coefficient is set to introduce the influence of the time dimension. By performing an exponential attenuation operation on the time interval of the state transition, the transition probability is adjusted so that the contribution of the state transition at a farther time point to the current state prediction gradually decreases. The attenuation operation can accurately describe the dynamic characteristics of the state changing over time, and finally generate a decoding state transition matrix. At the same time, the features of each dimension in the audio cycle feature vector are integrated to construct a P-dimensional feature vector. The feature covariance data is generated by calculating the inner product and mean of these feature vectors. Covariance data reflects the correlation between the features of each dimension and is the key to dimensionality reduction operations. A symmetric matrix is ​​constructed using the feature covariance data, and the feature decomposition data is obtained by solving the eigenvalue and decomposing the eigenvector on the matrix. The eigenvalues ​​and eigenvectors in the feature decomposition data respectively reflect the data variance size in different directions and the corresponding principal component direction. The eigenvalues ​​in the feature decomposition data are arranged in descending order, and the eigenvectors corresponding to the first several larger eigenvalues ​​are selected to generate the principal component feature data. The principal component feature data represents the most important information dimension in the original data. The projection matrix is ​​constructed by combining these principal component features in a column vector manner, and the projection matrix is ​​used to perform a linear mapping operation on the audio feature data to generate the dimension reduction vector data. The dimension reduction vector data is combined with the decoding state transfer matrix, and the state prediction calculation is performed using the maximum a posteriori probability criterion. Through probability calculation, the state corresponding to the maximum a posteriori probability is selected as the decoding prediction data, and the accuracy and rationality of the decoding prediction are ensured through quantitative probability analysis. The decoding prediction data is input into the preset task division model. In the task division model, the dependency graph of the decoding unit is constructed according to the decoding prediction data. The dependency graph reflects the dependency structure and priority between the decoding tasks. According to the dependency graph, the decoding processing sequence is divided to optimize the task execution order. By organizing the decoding task data according to the optimized sequence, more efficient parallel processing and resource allocation are achieved in the multi-mode SoC main control chip, generating high-quality decoding task data.

[0036] The decoded prediction data is input into the state detection unit, and the difference of the predicted state of consecutive frames is calculated to quantify the change range between each frame. By comparing the difference of the predicted state of consecutive frames with the preset difference threshold, the location with drastic changes can be effectively identified, and the decoding sequence is segmented at these locations to generate segmented sequence data. The segmentation result reflects the significance of signal changes in different time periods during the decoding process. The corresponding processing complexity parameter and time limit parameter are calculated for each data segment in the segmented sequence data. The processing complexity parameter is defined based on the signal feature dimension, feature change rate and required computing resources in the segment, while the time limit parameter reflects the real-time requirements of the decoding task. In order to quantify the weight of these parameters in task division, the processing complexity parameter, time limit parameter and data segment length parameter are linearly combined to generate task label data. The task label data is input into the graph construction module as the node weight, and directed connections are added between nodes according to the dependency relationship of different data segments in the segmented sequence to generate an adjacency matrix, thereby constructing the dependency graph of the decoding task. The graph structure intuitively describes the order and dependency characteristics between tasks. Each node represents a specific decoding task, and the directed edges between nodes represent the execution order and data transfer relationship of the tasks. The construction of the dependency graph ensures that the allocation of decoding tasks can meet the logical order and computing requirements of the decoding tasks to the greatest extent. In order to optimize task scheduling, the in-degree value and out-degree value of each node in the dependency graph are calculated, where the in-degree value represents the number of dependent tasks of the node, and the out-degree value represents the number of subsequent tasks affected by it. All nodes are arranged in descending order according to the in-degree value to form a processing queue, and node sequence data is generated accordingly. The node sequence data provides the priority order of task processing to ensure that high-priority tasks are allocated resources for processing first. Based on the node sequence data, the processing path is marked and the delay accumulation calculation is performed on each path. The path delay is the sum of the calculation delays of all task nodes along the path, which can reflect the execution efficiency of a path in the entire decoding process. For paths greater than the preset delay threshold, they are classified into the critical path set, and the task nodes are assigned processing core numbers in the critical path set. They are grouped according to the computing power of the processing core to generate task grouping data. Rearrange the task grouping data in chronological order to optimize the scheduling of decoding tasks. At the same time, perform statistical calculations on the task load on each processing core to analyze the working status and load distribution of the current processing core. Based on the load statistical data, set the balancing factor, and dynamically adjust the tasks that exceed the balancing threshold to reallocate them to the idle processing core, thereby optimizing resource utilization and improving parallel processing efficiency. The decoding task data is obtained through the above process.

[0037] Step 400: Perform hidden Markov modeling on the decoding task data, and obtain decoding optimization data by calculating through an improved Viterbi algorithm;

[0038] Specifically, the decoding task data is constructed as a state vector, which describes the distribution of different tasks in the decoding process in time series. By analyzing the distribution of the processing core of the task data, the state vector is modeled in time series to generate an initial transfer matrix. Each element of the initial transfer matrix represents the frequency or possibility of transition between states, reflecting the dynamic change law of the decoding task. In order to make the transition probability matrix mathematically operable, each element of the initial transfer matrix is ​​normalized. The sum of the elements in each row of the matrix is ​​normalized to 1 to generate a Markov transfer matrix. At the same time, the fundamental frequency information, energy information and zero-crossing information in the audio period feature vector are weighted and summed to generate feature weight data. Based on the feature weight data, the observation probability distribution is calculated, and these probabilities are used as the observation probability of the audio signal in different states. At the same time, in order to improve the reliability and practical significance of the observation probability, the feature weight data is used as a weighting coefficient to adjust the observation probability to generate a weighted observation matrix. The observation probability value in the weighted observation matrix is ​​multiplied by the state transition probability value in the Markov transfer matrix to generate a transmission probability matrix. The transmission probability matrix represents the joint probability of transmitting from a certain state to the next state. Based on the emission probability matrix, a probability threshold is set to delete the state paths below the target value. This pruning operation effectively reduces the complexity and computational burden of the model by removing unlikely state transition paths. For the remaining paths, their probability values ​​are renormalized to generate a pruned probability matrix. The pruned probability matrix is ​​decoded by the improved Viterbi algorithm. The pruned probability matrix is ​​input into the state decoding unit to calculate the local maximum a posteriori probability for each observation sequence. During the calculation process, the improved Viterbi algorithm introduces a dynamic constraint mechanism to ensure that a better path is selected first in the probability accumulation operation to avoid excessive path expansion or inefficient calculation in traditional algorithms. The cumulative probability sequence of the current observation sequence is generated by recursively calculating the cumulative probability sequence of each time step. After the calculation of the cumulative probability sequence is completed, the maximum probability path backtracking operation is performed to find the path with the highest cumulative probability through backtracking and reconstruct the state transition chain. The reconstruction process is based on the state sequence optimization of the hidden Markov model to ensure that the final output decoding optimization data has the highest global probability. After the reconstruction is completed, the decoding optimization data can accurately reflect the timing distribution and state transition law of the decoding task.

[0039] Step 500: Map the decoded optimization data to the hardware acceleration unit of the multi-mode SoC master chip, perform parallel decoding operations, and obtain target decoded data.

[0040] Specifically, the decoding optimization data is classified according to its computational complexity. By analyzing the computational resources required for the task, the computationally intensive matrix operations are mapped to the dedicated matrix processing units in the multi-mode SoC main control chip. These units are specially optimized for large-scale matrix multiplication, inversion and other complex operations, which can significantly improve the computational efficiency. At the same time, the vector operation tasks are mapped to the SIMD (single instruction multiple data stream) processing unit in the chip to use its efficient parallel processing capabilities to perform vector addition and subtraction, dot product and other operations. For scalar operation tasks, they are mapped to the general processing unit in the chip, and the control flow intensive calculations are completed through its flexible processing logic. This task classification process generates operation mapping data. The operation mapping data is input into the DMA (direct memory access) controller to establish a data prefetch queue. The DMA controller transfers the data to be processed from the off-chip memory to the on-chip cache, and opens up a separate cache area in the cache to store the data, ensuring that the data can be quickly accessed by the hardware acceleration unit and generating a data prefetch sequence. In pipeline processing, the data prefetch sequence is divided into four pipelines, including instruction prefetch, data loading, operation execution and result writeback. These pipelines are executed in parallel, and through efficient task division and scheduling, the resources of the hardware acceleration unit are fully utilized to generate pipeline processing data. Based on the pipeline processing data, the workload of the task is distributed according to the computing power of each processing unit, and the initial working frequency and working voltage are set for each processing unit to generate unit configuration data. The real-time load status of each processing unit in the unit configuration data is sampled and detected, and the data throughput and energy consumption per unit time are counted to generate performance statistics. By inputting the performance statistics into the dynamic power management unit, the working frequency gear of the processing unit is dynamically adjusted according to the ratio of data throughput to energy consumption to optimize the balance between power consumption and performance, and generate clock configuration data. The clock configuration data is divided and processed to generate independent clock signals for each processing unit, so that it can run independently at different working frequencies. When the task is completed, the idle processing unit is turned off through clock gating technology to further reduce power consumption and generate timing control data. The timing control data is used as a synchronization signal to align and merge the calculation results output by each processing unit to ensure that the final decoding result has a unified timing and data format and generate the target decoding data.

[0041] In the embodiment of the present application, the audio feature data is extracted by the DSP processing unit, and the anti-noise ability of the decoding process is improved by combining the multi-frame smoothing processing mechanism, so that the feature extraction is more accurate and reliable. The principal component analysis method is used to reduce the dimension of the audio features, which effectively reduces data redundancy, reduces the computational complexity, and improves the efficiency of feature extraction. The improved hidden Markov model and Viterbi algorithm are introduced, and the prediction accuracy of the state sequence is optimized by combining the audio cycle feature vector as the weight coefficient. The dynamic task partitioning model and the dependency graph are constructed to achieve the reasonable allocation of decoding tasks and improve the utilization rate of system resources. The data pipeline mechanism and the dynamic clock control strategy are adopted to achieve efficient parallel decoding processing. A complete performance evaluation and dynamic adjustment mechanism is established, and the system energy consumption is reduced by real-time monitoring and optimization strategies. The hardware acceleration unit and the intelligent power consumption management module are integrated to significantly improve the overall performance of the system. The adaptive optimization of the decoding process is realized, and the decoding strategy can be flexibly adjusted according to different audio data characteristics and processing requirements.

[0042] In a specific embodiment, the process of executing step 100 may specifically include the following steps:

[0043] The input audio data is segmented to obtain a plurality of audio data segments, and the plurality of audio data segments are discretely sampled and resampled according to a fixed number of sampling points to obtain framed audio data;

[0044] In the DSP processing unit, the sampling points of each frame of the audio data are squared and accumulated to obtain an accumulated result, and the accumulated result is divided by the number of sampling points of the corresponding frame to obtain energy data;

[0045] Perform sign bit judgment on the amplitudes of adjacent sampling points in each frame of audio data, and divide the total number of sign changes by the frame length to obtain zero-crossing data;

[0046] For each frame of the audio data, the product accumulation value of the delayed signal and the original signal is calculated according to the preset maximum delay point number to obtain initial autocorrelation data, and the calculated value of each delay point in the initial autocorrelation data is divided by the autocorrelation value of the zero delay point to obtain autocorrelation data;

[0047] A feature mapping relationship is established according to energy data, zero-crossing data and autocorrelation data to obtain audio feature data.

[0048] Specifically, the input continuous audio data is segmented according to the time axis, and the sliding window technology is used to divide the audio signal into multiple audio data segments with fixed lengths. The length of each segment is set to (unit is sampling point), and set the length of the overlapping part to The signal segment after segmentation is recorded as ,in Indicates the segment number, Indicates the sampling point index within the segment. This process ensures that the time domain information of the audio signal is fully described in a continuous time window. After segmentation, for each audio data segment Perform discrete sampling and resampling. Assume that the original sampling rate is , the target sampling rate is , the resampling process is achieved through interpolation or decimation algorithms. , then interpolation is required to increase the sampling rate by inserting new sampling points; if , the sampling rate is reduced by reducing the sampling points. The resampled signal is recorded as ,in Indicates the new index after resampling. Through this process, the audio data is standardized to a uniform sampling rate and framed audio data is generated. In the DSP processing unit, each frame of framed audio data is processed. Perform energy calculation. Perform square operation on each sampling point to calculate its instantaneous energy. The expression is:

[0049] ;

[0050] in, For Frame Middle The instantaneous energy of all sampling points is accumulated to get the total energy of the frame:

[0051] ;

[0052] in, Indicates the total number of sampling points in the frame. In order to obtain the average energy of the frame, the total energy Divide by the number of sampling points :

[0053] ;

[0054] in Representation frame The average energy of the frame is used to reflect the overall strength information of the frame. At the same time, the audio data of each frame is divided into The adjacent sampling points of the signal are used to perform sign bit judgment and count the number of times the signal crosses zero. The formula for sign bit judgment is:

[0055] ;

[0056] in, Representation frame No. Whether there is a sign change (i.e., zero crossing) between the sampling point and the previous sampling point. Accumulate and get the number of zero crossings :

[0057] ;

[0058] For normalization, the number of zero crossings is Divide by the frame length , and the zero-crossing rate is obtained:

[0059] ;

[0060] in, Indicates that the signal is in the frame The zero-crossing frequency in reflects the spectral characteristics of the signal. Calculate its autocorrelation characteristics. The autocorrelation calculation is achieved by multiplying the signal with its delayed version and accumulating it. The formula is:

[0061] ;

[0062] in, Representation frame In Delay The autocorrelation value at The value range is ,in is the preset maximum number of delay points. For standardization, the autocorrelation value of each delay point is divided by the autocorrelation value of the zero delay point. :

[0063] ;

[0064] in, Represents the normalized autocorrelation value, reflecting the similarity of the signal at different delays. , zero-crossing data and autocorrelation data Establish feature mapping relationship. Assume that the feature mapping formula is:

[0065] ;

[0066] in, For Frame Comprehensive audio features of , , is the weight factor of energy, zero crossing and autocorrelation characteristics, reflecting the importance of different features in the mapping relationship. By adjusting the weight factor, different audio characteristics can be emphasized according to actual needs.

[0067] In a specific embodiment, the process of executing step 200 may specifically include the following steps:

[0068] The autocorrelation data in the audio feature data is input into the local extreme value detection unit, the amplitude difference of adjacent data points is calculated and the position coordinates of the peaks and troughs are determined to obtain a periodic point sequence;

[0069] Performing a difference operation on the time coordinates of adjacent wave peak positions in the periodic point sequence to obtain a difference operation result, and converting the difference operation result into baseband domain data to obtain baseband sequence data;

[0070] The fundamental frequency sequence data is grouped and statistical features are calculated to obtain fundamental frequency statistical data, and linear interpolation is performed on the fundamental frequency statistical data to obtain fundamental frequency smoothing data;

[0071] Align the energy data in the audio feature data with the fundamental frequency smoothing data according to the time index, and perform attenuation on the data points in the fundamental frequency smoothing data whose corresponding energy values ​​are less than a preset energy threshold to obtain energy correction data;

[0072] Aligning the zero-crossing data in the audio feature data with the energy correction data according to the time index, performing weighted suppression on the data points in the energy correction data whose corresponding zero-crossing rates exceed a preset range, and obtaining feature correction data;

[0073] The feature correction data is input into a multi-frame joint filtering unit, and a sliding window is set to perform recursive filtering calculation on the continuous multi-frame data to obtain time series smoothed data;

[0074] A periodic feature index structure is constructed based on time series smoothing data, and the fundamental frequency information, energy weight and zero-crossing mark combination are mapped into an audio periodic feature vector.

[0075] Specifically, the autocorrelation data in the audio feature data Input local extreme value detection unit, which is used to detect the location of peaks and troughs. Autocorrelation data It is obtained by calculating the correlation between the delayed signal and the original signal and is defined as:

[0076] ;

[0077] in, It's a delay. is the signal frame length, Represents the original signal. By calculating the amplitude difference, using the formula:

[0078] ;

[0079] Compare adjacent points and detect Head The position of the extreme point is the peak, and vice versa. Record the time coordinates of these extreme points , forming a periodic point sequence ,in is the total number of peaks or troughs. and Calculate the time difference, the formula is:

[0080] ;

[0081] in, Represents the time interval between adjacent peaks, that is, the period of the signal. Convert to frequency:

[0082] ;

[0083] Get baseband frequency domain data , the sequence it constitutes is the base frequency sequence data For the base frequency sequence data, grouping is performed to extract statistical features. Assume that each group contains frequency values, calculate the average value of each group by grouping ,variance and median , the formula is:

[0084]

[0085] in is the group number. After statistical calculation, if some groups have missing data, they are supplemented by linear interpolation. The formula is:

[0086] ;

[0087] After interpolation, the smoothed fundamental frequency data is obtained . The energy data in the audio feature data Align the time index with the smoothed fundamental frequency data. During the alignment process, the energy value corresponding to the fundamental frequency smoothed data is less than the preset energy threshold. The data points are subjected to attenuation processing, and the attenuation formula is:

[0088] ;

[0089] in, is the attenuation coefficient, adjust the fundamental frequency value of the low energy data point, and obtain the energy correction data. Align the time index with the energy correction data and calculate the corresponding zero-crossing rate in the energy correction data. Out of preset range The weighted suppression is performed on the data points, and the suppression formula is:

[0090] ;

[0091] in, is the weighted suppression coefficient. The feature correction data is input into the multi-frame joint filtering unit, and the sliding window is used to recursively filter the continuous multi-frame data. The formula is:

[0092]

[0093] in is the sliding window size, It is After recursive filtering, time series smoothing data is generated . Based on the time series smoothing data, a periodic feature index structure is constructed, and the smoothed fundamental frequency information is , Energy Weight and zero crossing marker The combined map is an audio period feature vector. Assume that the feature vector is in the form of:

[0094] ;

[0095] in, It is the audio period feature vector, which integrates the period, energy and frequency characteristics of the signal.

[0096] In a specific embodiment, the process of executing step 300 may specifically include the following steps:

[0097] Setting a state quantity threshold and a time window length for the audio cycle feature vector, performing state quantization classification on the audio cycle feature vector in each time window, and obtaining state sequence data;

[0098] Input the state sequence data into the state counting matrix, record the number of transitions between adjacent states, and perform probability normalization operation on each element of the state counting matrix to obtain the probability initial matrix;

[0099] The time series attenuation coefficient is set for the initial probability matrix, and the transfer probability is exponentially attenuated according to the time interval of the state transfer to obtain the decoding state transfer matrix;

[0100] The features of each dimension of the audio feature data are constructed as a P-dimensional feature vector, and the inner product and mean of the P×P-dimensional feature vector are calculated to obtain feature covariance data. At the same time, a symmetric matrix is ​​constructed for the feature covariance data, and the eigenvalue solution and eigenvector decomposition operations are performed on the symmetric matrix to obtain feature decomposition data;

[0101] Arrange the eigenvalues ​​in the feature decomposition data in descending order to obtain principal component feature data, and combine the principal component feature data in a column vector manner to construct a projection matrix, perform a linear mapping operation on the audio feature data, and obtain dimension-reduced vector data;

[0102] Perform state prediction calculation on the dimension-reduced vector data and the decoding state transfer matrix, and select the state corresponding to the maximum a posteriori probability as the decoding prediction data;

[0103] The decoding prediction data is input into a preset task division model, a dependency graph of the decoding unit is constructed, and the decoding processing sequence is divided according to the dependency graph to obtain the decoding task data.

[0104] Specifically, for the audio period feature vector Set the number of status thresholds and time window length , quantize the dimensional features of the feature vector according to the preset discretization rules. The quantization formula is expressed as:

[0105] ;

[0106] in, is the quantization step size, Representation characteristics The corresponding quantized state value. Through this quantization process, the feature vector is Classified as These states are organized in time series to form state sequence data. ,in is the length of the state sequence. Input the state sequence data into the state count matrix and record the number of transitions between adjacent states. The elements represent the state To status The number of transfers is defined as:

[0107]

[0108] in, is the Kronecker function, when When the state count matrix is ​​completed, it is normalized to obtain the probability initial matrix :

[0109]

[0110] After normalization, Indicates from the state Transfer to state The conditional probability of . For the initial probability matrix Set the timing decay factor , and according to the time interval of state transfer Perform an exponential decay operation on the transition probability to generate the decoding state transition matrix :

[0111] ;

[0112] Among them, the exponential decay function It is used to reduce the weight of long-distance state transfer, so as to better conform to the timing characteristics of audio signals. Build as dimensional feature vector , and calculate the inner product and mean between the eigenvectors to construct the eigencovariance matrix :

[0113]

[0114] in, It is a feature The mean of . The feature covariance matrix is a symmetric matrix that reflects the correlation between the features. Perform eigenvalue solution and eigenvector decomposition to obtain eigendecomposition data and the corresponding set of eigenvectors Arrange the eigenvalues ​​in descending order and select the first The eigenvectors corresponding to the largest eigenvalues ​​construct the principal component feature data and combine them into a projection matrix in the form of column vectors. :

[0115] ;

[0116] Use the projection matrix to linearly map the original audio feature data to obtain the reduced dimension vector data :

[0117] ;

[0118] Reduce the dimension of the vector data and the decoding state transition matrix Combined, use the Bayesian rule to calculate the posterior probability of each state, and select the state corresponding to the maximum posterior probability as the decoding prediction data:

[0119] ;

[0120] in, is the observed data in state The decoding prediction data is input into the preset task division model, and the dependency graph of the decoding unit is constructed according to the task dependency. The nodes in the dependency graph represent the decoding tasks, and the edges represent the dependencies between tasks. By analyzing the dependency graph, the decoding processing sequence is divided according to the task priority, and the decoding task data is generated.

[0121] In this embodiment, a time series attenuation coefficient λ is set for the probability initial matrix, and an exponential decay operation is performed on the transfer probability according to the time interval of the state transfer to obtain a decoded state transfer matrix, including: establishing a time series analysis window for the probability initial matrix, calculating the timestamp sequence of each state transfer, and obtaining state time series data; dividing the state time series data into multiple sub-intervals according to the time span, calculating the local mean of the state transfer probability in each sub-interval, and obtaining interval probability data; calculating the probability change rate between adjacent sub-intervals for the interval probability data, and determining the value range of the time series attenuation coefficient λ according to the distribution characteristics of the change rate, and obtaining attenuation coefficient data; the attenuation coefficient data Substitute the exponential decay function, calculate the decay weights for different time intervals, and obtain the time series weight data; perform weighted product operations on the corresponding elements of the time series weight data and the probability initial matrix to obtain the time series weight matrix; construct a multi-layer state transition network according to the distance relationship of the state transition time based on the time series weight matrix, and mark the time dependency of the state transition in the network to obtain the time series network data; calculate the cumulative probability for each state transition path in the time series network data, and sort the transfer paths by importance according to the cumulative probability to obtain the path probability data; reconstruct the path probability data into a matrix form, and normalize the matrix elements to obtain the decoding state transition matrix.

[0122] In a specific embodiment, the execution step inputs the decoding prediction data into a preset task division model, constructs a dependency graph of the decoding unit, and divides the decoding processing sequence according to the dependency graph. The process of obtaining the decoding task data may specifically include the following steps:

[0123] Input the decoded prediction data into the state detection unit, calculate the difference of the prediction states of consecutive frames, and segment the positions where the difference is greater than a preset difference threshold to obtain segment sequence data;

[0124] Calculate the corresponding processing complexity parameter and time limit parameter for each data segment in the segmented sequence data, and linearly combine the processing complexity parameter and time limit parameter with the data segment length parameter to obtain task labeling data;

[0125] The task label data is input into the graph construction module as the node weight, and directed connections are added between nodes according to the data dependency relationship to construct the adjacency matrix and obtain the dependency graph;

[0126] Calculate the corresponding in-degree and out-degree values ​​for each node in the dependency graph, and construct a processing queue by arranging the nodes in descending order of in-degree values ​​to obtain node sequence data;

[0127] Mark the processing paths according to the node sequence data, and perform delay accumulation calculation on each path, add the paths with delays greater than the preset threshold to the critical path set, obtain path analysis data, assign processing core numbers to the task nodes in the path analysis data, and group the task nodes according to the computing power of the processing core to obtain task grouping data;

[0128] The task grouping data is reordered in chronological order, and the task load on each processing core is statistically calculated to obtain load statistical data. A balancing factor is set based on the load statistical data, and tasks with loads exceeding the balancing threshold are reallocated to idle processing cores to obtain decoding task data.

[0129] Specifically, the decoded prediction data The input state detection unit analyzes the predicted state changes of consecutive frames. By calculating the state difference between adjacent frames , the formula is:

[0130] ;

[0131] in, and Representation frame and frame The predicted state. All differences are compared with the preset difference threshold For comparison, if , it is considered that the state has changed significantly, and segmentation is performed at this position to generate segmented sequence data ,in is the number of segments, each segment Contains consecutive frame indices. For each data segment Calculation Processing Complexity Parameters and time limit parameters The processing complexity parameter is based on the number of features in the data segment. , average state change rate And the number of frames in the segment Defined as:

[0132] ;

[0133] Time limit parameters According to the real-time requirements, the start and end time index of the segment is used for calculation:

[0134] ;

[0135] Will handle complexity parameters , time limit parameters and data segment length parameters Perform linear combination to generate task labeled data :

[0136] ;

[0137] in, is the weight factor, which is used to adjust the impact of each parameter on the task mark. Input graph building module, and add directed connections between nodes based on the dependencies between data segments to generate adjacency matrix , whose elements are defined as:

[0138] ;

[0139] Graph composed of adjacency matrices Represents a dependency graph, where is a set of task nodes, is a set of dependency edges. Each node in Calculate its indegree and out :

[0140]

[0141] The nodes are sorted according to their in-degree values. Sort in descending order and build a processing queue , forming node sequence data. The processing path is marked according to the node sequence data, and the delay of each path is accumulated and calculated. It is defined as the sum of the time limit parameters of all tasks on the path:

[0142] ;

[0143] Will be greater than the preset delay threshold Add the path to the key path collection . Assign processing core numbers to task nodes in the critical path set , according to the computing power of each processing core The task nodes are grouped according to the following principles:

[0144] ;

[0145] Generate task group data . Rearrange the task group data in chronological order to ensure that the execution of the decoding task meets the timing constraints. Count the task load on each processing core and calculate the number of tasks per unit time. :

[0146] ;

[0147] in, is the execution time window of the task. Set the balancing factor based on load statistics , when the load of a core exceeds the balancing threshold When the overloaded tasks are redistributed to the idle processing cores , the adjusted task allocation satisfies:

[0148]

[0149] Generate optimized decoding task data.

[0150] In a specific embodiment, the process of executing step 400 may specifically include the following steps:

[0151] The decoding task data is constructed as a state vector, and the state vector is modeled in time series according to the distribution of processing cores to generate an initial transfer matrix;

[0152] Normalize each element in the initial transfer matrix, normalize the sum of each row of elements to 1, and generate a Markov transfer matrix;

[0153] Performing a weighted sum operation on the fundamental frequency information, energy information and zero-crossing information of the audio period feature vector to obtain feature weight data, calculating the observation probability distribution for the feature weight data, and using the feature weight data as a weighting coefficient to adjust the observation probability to obtain a weighted observation matrix;

[0154] The observation probability value in the weighted observation matrix is ​​multiplied by the state transition probability value of the Markov transfer matrix to obtain the emission probability matrix, and the probability threshold is set based on the emission probability matrix, the state path with a probability value less than the target value is deleted, and the probability values ​​of the remaining paths are re-normalized to obtain the pruning probability matrix;

[0155] The pruned probability matrix is ​​input into the state decoding unit through the improved Viterbi algorithm, the local maximum a posteriori probability is calculated for each observation sequence, and the probability multiplication operation is performed to obtain the cumulative probability sequence, and the maximum probability path backtracking and state transfer chain reconstruction are performed on the cumulative probability sequence to obtain the decoding optimization data.

[0156] Specifically, the decoding task data is organized into a state vector ,in Indicates that the decoding task is The states at each time point correspond to different processing core allocations, task priorities, or specific decoding operation categories. Based on these state vectors, a timing model is established according to the time sequence of task execution. Timing modeling generates an initial transition matrix by calculating the number of transitions between states at adjacent time points. , where the matrix elements Indicates from the state Transfer to state The frequency is defined as:

[0157]

[0158] in, is the Kronecker function, when The value is 1 when , otherwise it is 0. Each row of is normalized to generate a Markov transfer matrix The normalization operation ensures that the sum of the elements in each row is 1, so that the matrix can represent the conditional probability of state transition:

[0159]

[0160] in, Indicates from the state Transfer to state The probability of is the total number of states. The fundamental frequency information of the audio period feature vector , Energy Information and zero-crossing information Perform weighted summation to comprehensively reflect the characteristic weight data of the audio signal Assume that the weight coefficient is , β, γ, the calculation formula of the feature weight data is:

[0161] ;

[0162] in, It is a weight parameter used to adjust the feature contribution. Based on the feature weight data , calculate the observed probability distribution , that is, the probability of observing a specific feature in different states. The feature weight data is used as the weighting coefficient to adjust the observation probability distribution and generate a weighted observation matrix :

[0163] ;

[0164] in, Indicates in status The observed features The weighted probability of . The observed probability value and Markov transition matrix The state transition probability values ​​are multiplied element by element to generate the emission probability matrix :

[0165] ;

[0166] The emission probability matrix describes the joint probability of state transition and feature observation. On this basis, the probability threshold is set , prune the emission probability matrix, delete the probability paths that are less than the threshold, and renormalize the probability values ​​of the remaining paths to generate a pruned probability matrix :

[0167]

[0168] The pruned probability matrix optimizes the computational complexity of state transitions while retaining important transition paths. The pruned probability matrix is ​​decoded using the improved Viterbi algorithm. The Viterbi algorithm uses a dynamic programming method to calculate the local maximum a posteriori probability of each observation sequence. Assume Indicates at time Time Status The maximum cumulative probability, then the recursive formula is:

[0169] ;

[0170] The cumulative probability sequence is obtained by recursive calculation, and the maximum probability path backtracking is performed at the final time step:

[0171] ;

[0172] in, is the decoding path corresponding to the maximum a posteriori probability. The state transfer chain is reconstructed by path backtracking, and finally the decoding optimization data is generated.

[0173] In a specific embodiment, the process of executing step 500 may specifically include the following steps:

[0174] The decoding optimization data is task-classified according to computational complexity, and computationally intensive matrix operations are mapped to dedicated matrix processing units in the multi-mode SoC main control chip, vector operations are mapped to SIMD processing units in the multi-mode SoC main control chip, and scalar operations are mapped to general-purpose processing units in the multi-mode SoC main control chip to obtain operation mapping data;

[0175] A data pre-fetch queue is established in the DMA controller for the operation mapping data, and an independent cache area is opened in the on-chip cache to store the data to be processed, so as to obtain a data pre-fetch sequence;

[0176] Perform four-stage pipeline division on the data prefetch sequence, set instruction prefetch, data loading, operation execution and result write-back as four pipeline stages executed in parallel, and obtain pipeline processing data;

[0177] Allocate the workload of the pipeline processing data according to the computing capacity of the processing unit, and set the initial operating frequency and operating voltage of each processing unit to obtain unit configuration data;

[0178] Sampling and detecting the real-time load status of each processing unit in the unit configuration data, and counting the data throughput and energy consumption per unit time to obtain performance statistics;

[0179] Inputting the performance statistics data into the dynamic power management unit, adjusting the operating frequency gear according to the ratio of data throughput and energy consumption, and obtaining clock configuration data;

[0180] The clock configuration data is divided into different frequencies to generate independent clock signals for each processing unit. Clock gating is performed when the task is completed to obtain timing control data. The timing control data is used as a synchronization signal to perform data alignment and result merging on the calculation results output by each processing unit to obtain the target decoded data.

[0181] Specifically, the computational complexity of the decoding optimization data is analyzed, and the tasks are divided into three categories: matrix operations, vector operations, and scalar operations. The complexity of matrix operations is generally high, mainly involving large-scale matrix multiplication, matrix inversion, and other operations. Indicates that is the dimension of the matrix. Such tasks are mapped to the dedicated matrix processing units in the multi-mode SoC master chip. These units usually support hardware acceleration, such as pipeline matrix multipliers or tensor cores to achieve efficient operations. Vector operations include vector addition and subtraction, dot product, etc., with a complexity of , suitable for mapping to SIMD (single instruction multiple data) processing units, using their parallel capabilities to improve efficiency. Scalar operations have the lowest complexity and are suitable for mapping to general-purpose processing units for execution. These units usually have more flexible logic to handle control-intensive tasks. Through this classification and mapping process, operation mapping data is generated. The operation mapping data is input into the DMA (direct memory access) controller to establish a data prefetch queue. The DMA controller transfers the data to be processed from the off-chip memory to the on-chip cache by managing data transfer. And in the on-chip cache, independent cache areas are opened for different task types to store the data required for matrix operations, vector operations and scalar operations, thereby generating a data prefetch sequence. After the data preparation is completed, the data prefetch sequence is divided into four pipeline stages, including instruction prefetch, data loading, operation execution and result writeback. These pipeline stages are executed in parallel. For example, in one pipeline cycle, one part of the processing unit is loading the data of the next task, another part is executing the current task, and the third part is writing back the results of the completed task. The performance of the pipeline is expressed by its cycle as:

[0182] ;

[0183] in, They represent the time for instruction prefetching, data loading, operation execution, and result writing back. Based on the processing results of the pipeline, the pipeline processing data is divided into the computing power of the processing unit. Allocate workload based on task complexity satisfy At the same time, set the initial operating frequency for each processing unit and operating voltage , and its relationship is approximately expressed by the dynamic voltage frequency scaling (DVFS) formula:

[0184] ;

[0185] in, is the power consumption, is a hardware constant. By properly setting and , while meeting performance requirements and reducing power consumption, generate unit configuration data. During operation, perform real-time load state sampling and detection on each processing unit in the unit configuration data, and count the data throughput per unit time. and energy consumption , respectively defined as:

[0186]

[0187] Performance statistics Enter the dynamic power management unit by comparing the ratio of throughput to energy consumption , adjust the working frequency gear To optimize the balance between performance and energy consumption. Adjusted clock configuration data and Through frequency division, an independent clock signal is generated for each processing unit, and clock gating technology is used to shut down idle units when the task is completed, thereby further reducing power consumption. As a synchronization signal, it is used to perform data alignment and result merging on the calculation results of each processing unit. Data alignment ensures that the outputs of different processing units have consistent timing, while result merging integrates the outputs of all tasks into the final target decoded data.

[0188] The above describes the audio decoding performance optimization method of the multi-mode SoC main control chip in the embodiment of the present application. The following describes the audio decoding performance optimization device 10 of the multi-mode SoC main control chip in the embodiment of the present application. Figure 2 In the embodiment of the present application, an embodiment of the audio decoding performance optimization device 10 of the multi-mode SoC main control chip includes:

[0189] The frame processing module 11 is used to perform frame processing on the input audio data through the DSP processing unit and extract audio feature data;

[0190] A period detection module 12 is used to perform period detection and multi-frame smoothing processing on the audio feature data to obtain an audio period feature vector;

[0191] The sequence analysis module 13 is used to perform principal component transformation and feature dimension reduction based on the audio period feature vector to obtain decoding prediction data, and input the decoding prediction data into a preset task division model to perform decoding processing sequence analysis to obtain decoding task data;

[0192] A modeling module 14 is used to perform hidden Markov modeling on the decoding task data and obtain decoding optimization data by calculating the improved Viterbi algorithm;

[0193] The decoding operation module 15 is used to map the decoding optimization data to the hardware acceleration unit of the multi-mode SoC main control chip, perform parallel decoding operations, and obtain target decoding data.

[0194] Through the cooperation of the above components, the audio feature data is extracted through the DSP processing unit, and the multi-frame smoothing processing mechanism is combined to improve the anti-noise ability of the decoding process, making the feature extraction more accurate and reliable. The principal component analysis method is used to reduce the dimension of the audio features, which effectively reduces data redundancy, reduces the computational complexity, and improves the efficiency of feature extraction. The improved hidden Markov model and Viterbi algorithm are introduced, and the prediction accuracy of the state sequence is optimized by combining the audio period feature vector as the weight coefficient. The dynamic task partitioning model and dependency graph are constructed to achieve the reasonable allocation of decoding tasks and improve the utilization of system resources. The data pipeline mechanism and dynamic clock control strategy are adopted to achieve efficient parallel decoding processing. A complete performance evaluation and dynamic adjustment mechanism is established, and the system energy consumption is reduced through real-time monitoring and optimization strategies. The hardware acceleration unit and the intelligent power consumption management module are integrated to significantly improve the overall performance of the system. The adaptive optimization of the decoding process is realized, and the decoding strategy can be flexibly adjusted according to different audio data characteristics and processing requirements.

[0195] See also Figure 3 , Figure 3 This is a schematic block diagram of the structure of an electronic device 300 provided in an embodiment of the present application. The electronic device 300 includes a processor 301 and a memory 302. The processor 301 and the memory 302 are connected via a device bus 303, wherein the memory 302 may include a non-volatile storage medium and an internal memory.

[0196] The non-volatile storage medium can store a computer program. The computer program includes program instructions, and when the program instructions are executed by the processor 301, the processor 301 can execute any of the above-mentioned audio decoding performance optimization methods for the multi-mode SoC main control chip.

[0197] The processor 301 is used to provide computing and control capabilities to support the operation of the entire electronic device 300 .

[0198] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor 301, the processor 301 can execute any of the above-mentioned methods for optimizing the audio decoding performance of the multi-mode SoC master control chip.

[0199] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a partial structure related to the present application scheme, and does not constitute a limitation on the electronic device 300 involved in the present application scheme. The specific electronic device 300 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0200] It should be understood that the processor 301 may be a central processing unit (CPU), and the processor 301 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0201] It should be noted that technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of the electronic device 300 described above can refer to the corresponding process of the audio decoding performance optimization method of the aforementioned multi-mode SoC master chip, and will not be repeated here.

[0202] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by one or more processors, the one or more processors implement the audio decoding performance optimization method of the multi-mode SoC master control chip provided in the embodiment of the present application.

[0203] The computer-readable storage medium may be an internal storage unit of the electronic device 300 in the aforementioned embodiment, such as a hard disk or memory of the electronic device 300. The computer-readable storage medium may also be an external storage device of the electronic device 300, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped with the electronic device 300.

[0204] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0205] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable an electronic device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0206] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for optimizing the audio decoding performance of a multi-mode SoC master chip, characterized in that: The method comprises: The input audio data is framed by a DSP processing unit, and audio feature data is extracted; specifically, the input audio data is segmented to obtain a plurality of audio data segments, and the plurality of audio data segments are discretely sampled and resampled according to a fixed number of sampling points to obtain framed audio data; in the DSP processing unit, the sampling points of the framed audio data of each frame are squared and accumulated to obtain an accumulated result, and the accumulated result is divided by the number of sampling points of the corresponding frame to obtain energy data; the amplitude of adjacent sampling points in the framed audio data of each frame is subjected to sign bit judgment, and the total number of sign changes is divided by the frame length to obtain zero-crossing data; for each frame of the framed audio data, the product accumulation value of the delayed signal and the original signal is calculated according to a preset maximum number of delay points to obtain initial autocorrelation data, and the calculated value of each delay point in the initial autocorrelation data is divided by the autocorrelation value of the zero delay point to obtain autocorrelation data; a feature mapping relationship is established according to the energy data, the zero-crossing data and the autocorrelation data to obtain audio feature data; Performing period detection and multi-frame smoothing processing on the audio feature data to obtain an audio period feature vector; Performing principal component transformation and feature dimension reduction based on the audio period feature vector to obtain decoding prediction data, and inputting the decoding prediction data into a preset task division model to perform decoding processing sequence analysis to obtain decoding task data; Performing hidden Markov modeling on the decoding task data, and obtaining decoding optimization data by calculating through an improved Viterbi algorithm; The decoding optimization data is mapped to the hardware acceleration unit of the multi-mode SoC main control chip, and parallel decoding operations are performed to obtain target decoding data.

2. The method for optimizing the audio decoding performance of a multi-mode SoC master chip according to claim 1, characterized in that: The performing period detection and multi-frame smoothing processing on the audio feature data to obtain an audio period feature vector includes: Inputting the autocorrelation data in the audio feature data into a local extreme value detection unit, calculating the amplitude difference of adjacent data points and determining the position coordinates of the peaks and valleys, and obtaining a periodic point sequence; Performing a difference operation on the time coordinates of adjacent wave peak positions in the periodic point sequence to obtain a difference operation result, and converting the difference operation result into baseband domain data to obtain baseband sequence data; The fundamental frequency sequence data is grouped and statistical features are calculated to obtain fundamental frequency statistical data, and the fundamental frequency statistical data is linearly interpolated and supplemented to obtain fundamental frequency smoothed data; Aligning the energy data in the audio feature data with the fundamental frequency smoothing data according to the time index, and performing attenuation on the data points in the fundamental frequency smoothing data whose corresponding energy values ​​are less than a preset energy threshold, to obtain energy correction data; Aligning the zero-crossing data in the audio feature data with the energy correction data according to the time index, performing weighted suppression on data points in the energy correction data whose corresponding zero-crossing rates exceed a preset range, to obtain feature correction data; Inputting the feature correction data into a multi-frame joint filtering unit, setting a sliding window to perform recursive filtering calculation on the continuous multi-frame data to obtain time series smoothed data; A periodic feature index structure is constructed based on the time series smoothing data, and the fundamental frequency information, energy weight and zero-crossing mark combination are mapped into an audio periodic feature vector.

3. The method for optimizing the audio decoding performance of a multi-mode SoC master chip according to claim 2, characterized in that: The method of performing principal component transformation and feature dimension reduction based on the audio period feature vector to obtain decoding prediction data, and inputting the decoding prediction data into a preset task division model to perform decoding processing sequence analysis to obtain decoding task data includes: Setting a state quantity threshold and a time window length for the audio cycle feature vector, and performing state quantization classification on the audio cycle feature vector in each time window to obtain state sequence data; Inputting the state sequence data into a state counting matrix, recording the number of transitions between adjacent states, and performing a probability normalization operation on each element of the state counting matrix to obtain a probability initial matrix; Setting a time series decay coefficient for the initial probability matrix, and performing an exponential decay operation on the transition probability according to the time interval of the state transition, to obtain a decoding state transition matrix; Constructing the features of each dimension of the audio feature data into a P-dimensional feature vector, and calculating the inner product and mean of the P×P-dimensional feature vector to obtain feature covariance data, and at the same time, constructing a symmetric matrix for the feature covariance data, and performing eigenvalue solution and eigenvector decomposition operations on the symmetric matrix to obtain feature decomposition data; Arrange the eigenvalues ​​in the eigendecomposition data in descending order to obtain principal component feature data, combine the principal component feature data in a column vector manner to construct a projection matrix, perform a linear mapping operation on the audio feature data, and obtain dimension-reduced vector data; Performing state prediction calculation on the reduced-dimensional vector data and the decoding state transfer matrix, and selecting the state corresponding to the maximum a posteriori probability as decoding prediction data; The decoding prediction data is input into a preset task division model, a dependency graph of the decoding unit is constructed, and a decoding processing sequence is divided according to the dependency graph to obtain decoding task data.

4. The method for optimizing the audio decoding performance of a multi-mode SoC master chip according to claim 3, characterized in that: The step of inputting the decoding prediction data into a preset task division model, constructing a dependency graph of a decoding unit, and dividing a decoding processing sequence according to the dependency graph to obtain decoding task data includes: Input the decoded prediction data into a state detection unit, calculate the difference of the prediction states of consecutive frames, and segment the positions where the difference is greater than a preset difference threshold to obtain segment sequence data; Calculating a corresponding processing complexity parameter and a time limit parameter for each data segment in the segmented sequence data, and linearly combining the processing complexity parameter and the time limit parameter with a data segment length parameter to obtain task marking data; Inputting the task label data as node weights into a graph construction module, and adding directed connections between nodes according to data dependencies, constructing an adjacency matrix, and obtaining a dependency graph; Calculate the corresponding in-degree value and out-degree value for each node in the dependency graph, and construct a processing queue by arranging the nodes in descending order of the in-degree values ​​to obtain node sequence data; Marking processing paths according to the node sequence data, and performing delay accumulation calculation on each path, adding paths with delays greater than a preset threshold to a critical path set to obtain path analysis data, assigning processing core numbers to task nodes in the path analysis data, and grouping the task nodes according to the computing power of the processing core to obtain task grouping data; The task grouping data is reordered in chronological order, and the task load on each processing core is statistically calculated to obtain load statistical data, and a balancing factor is set based on the load statistical data, and tasks with loads exceeding a balancing threshold are reallocated to an idle processing core to obtain decoding task data.

5. The method for optimizing the audio decoding performance of a multi-mode SoC master chip according to claim 4, characterized in that: The step of performing hidden Markov modeling on the decoding task data and obtaining decoding optimization data by calculating through an improved Viterbi algorithm includes: The decoding task data is constructed as a state vector, and the state vector is subjected to time series modeling according to the distribution of processing cores to generate an initial transfer matrix; Normalizing each element in the initial transfer matrix, normalizing the sum of the elements in each row to 1, and generating a Markov transfer matrix; Performing a weighted sum operation on the fundamental frequency information, energy information and zero-crossing information of the audio period feature vector to obtain feature weight data, calculating the observation probability distribution for the feature weight data, and using the feature weight data as a weighting coefficient to adjust the observation probability to obtain a weighted observation matrix; The observation probability values ​​in the weighted observation matrix are multiplied by the state transition probability values ​​of the Markov transfer matrix to obtain a transmission probability matrix, and a probability threshold is set based on the transmission probability matrix, and the state paths with probability values ​​less than the target value are deleted, and the probability values ​​of the remaining paths are re-normalized to obtain a pruning probability matrix; The pruned probability matrix is ​​input into the state decoding unit through the improved Viterbi algorithm, the local maximum a posteriori probability is calculated for each observation sequence, and the probability accumulation operation is performed to obtain a cumulative probability sequence, and the maximum probability path backtracking and state transfer chain reconstruction are performed on the cumulative probability sequence to obtain decoding optimization data.

6. The method for optimizing the audio decoding performance of a multi-mode SoC master chip according to claim 5, characterized in that: The method of mapping the decoding optimization data to the hardware acceleration unit of the multi-mode SoC main control chip, performing parallel decoding operations, and obtaining target decoding data includes: The decoding optimization data is task-classified according to computational complexity, computationally intensive matrix operations are mapped to dedicated matrix processing units in the multi-mode SoC main control chip, vector operations are mapped to SIMD processing units in the multi-mode SoC main control chip, and scalar operations are mapped to general-purpose processing units in the multi-mode SoC main control chip to obtain operation mapping data; The operation mapping data is used to establish a data pre-fetch queue in a DMA controller, and an independent cache area is opened in an on-chip cache to store the data to be processed, so as to obtain a data pre-fetch sequence; Performing four-stage pipeline division on the data prefetch sequence, setting instruction prefetch, data loading, operation execution and result write-back as four pipeline stages executed in parallel, and obtaining pipeline processing data; Allocating workloads to the pipeline processing data according to the computing capabilities of the processing units, and setting initial operating frequencies and operating voltages of the processing units to obtain unit configuration data; Sampling and detecting the real-time load status of each processing unit in the unit configuration data, and counting the data throughput and energy consumption per unit time to obtain performance statistical data; Inputting the performance statistical data into a dynamic power consumption management unit, adjusting the operating frequency gear according to the ratio of data throughput to energy consumption, and obtaining clock configuration data; The clock configuration data is frequency-divided to generate an independent clock signal for each processing unit, and clock gating is performed when the task is completed to obtain timing control data. The timing control data is used as a synchronization signal to perform data alignment and result merging on the calculation results output by each processing unit to obtain target decoded data.

7. An audio decoding performance optimization device for a multi-mode SoC main control chip, characterized in that: The method for optimizing the audio decoding performance of a multi-mode SoC main control chip according to any one of claims 1 to 6 is used, wherein the audio decoding performance optimization device of the multi-mode SoC main control chip comprises: The frame processing module is used to perform frame processing on the input audio data through the DSP processing unit and extract audio feature data; A period detection module, used for performing period detection and multi-frame smoothing processing on the audio feature data to obtain an audio period feature vector; A sequence analysis module, used for performing principal component transformation and feature dimension reduction based on the audio period feature vector to obtain decoding prediction data, and inputting the decoding prediction data into a preset task division model to perform decoding processing sequence analysis to obtain decoding task data; A modeling module, used for performing hidden Markov modeling on the decoding task data, and obtaining decoding optimization data by calculating through an improved Viterbi algorithm; The decoding operation module is used to map the decoding optimization data to the hardware acceleration unit of the multi-mode SoC main control chip, perform parallel decoding operations, and obtain target decoding data.

8. An electronic device, characterized in that: The electronic device comprises: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instruction in the memory so that the electronic device executes the audio decoding performance optimization method of the multi-mode SoC master control chip as described in any one of claims 1-6.

9. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instruction is executed by the processor, the audio decoding performance optimization method of the multi-mode SoC master control chip as described in any one of claims 1-6 is implemented.

Images