Audio data processing method and related device

Abstract

An embodiment of the present application provides an audio data processing method and related apparatus, the method comprising: obtaining a frame of audio data from audio data to be processed, the audio data to be processed being audio data obtained after collecting a sound source signal through a first channel; calculating a first autocorrelation matrix based on the frame of audio data; when the values on the diagonal of the first autocorrelation matrix are unequal, modifying the values on the diagonal of the first autocorrelation matrix to equal constants to obtain a second autocorrelation matrix; and performing dereverberation processing on the frame of audio data based on the second autocorrelation matrix to obtain processed audio data. By dereverberating the audio data to be processed according to the present application, the audio quality of the dereverberated audio signal can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and more particularly to an audio data processing method and related apparatus. Background Art

[0002] In everyday life, the sounds audible to the human ear are mechanical waves, also known as sound waves. As sound waves propagate through a propagation medium and reach the receiving end, they are often reflected and absorbed by obstacles. For example, when sound waves propagate indoors, they are reflected by obstacles such as walls, ceilings, and floors. With each reflection, a portion of the sound wave is absorbed by the obstacle. Even after the sound source ceases, the sound wave undergoes multiple reflections and absorptions within the room before disappearing. Therefore, multiple sound waves continue to mix for a period of time after the sound source ceases, indicating that sound persists even after the sound source ceases. This persistence is known as reverberation.

[0003] Due to reverberation, the audio signal received by a receiver, such as a microphone, includes not only the direct portion of the sound source but also reflected signals due to multipath propagation through the transmission channel. This results in poor audio signal quality. To improve the quality of the audio signal received by the receiver, dereverberation processing is required. Improving the audio quality of dereverberated audio signals is a topic of ongoing research in this field. Summary of the Invention

[0004] The embodiments of the present application provide an audio data processing method and related devices. By performing dereverberation on the audio data to be processed, the audio quality of the dereverberated audio signal can be improved.

[0005] In a first aspect, an embodiment of the present application provides an audio data processing method, comprising:

[0006] Acquire a frame of audio data from the audio data to be processed, where the audio data to be processed is audio data obtained by collecting a sound source signal through the first channel;

[0007] Calculating a first autocorrelation matrix based on the above-mentioned one frame of audio data;

[0008] In the case where the values on the diagonal of the first autocorrelation matrix are not equal, the values on the diagonal of the first autocorrelation matrix are modified to equal constants to obtain a second autocorrelation matrix;

[0009] De-reverberation processing is performed on the frame of audio data based on the second autocorrelation matrix to obtain processed audio data.

[0010] In an embodiment of the present application, for the above-mentioned frame of audio data, when the values on the diagonal of the calculated first autocorrelation matrix are not equal, the values on the diagonal of the first autocorrelation matrix are modified to equal constants. Since the diagonal in the autocorrelation matrix reflects the autocorrelation characteristics, the different values on the diagonal of the autocorrelation matrix mean that the autocorrelation matrix includes the influence of the first channel itself. In the process of dereverberation processing of the above-mentioned frame of audio data, the values on the diagonal of the first autocorrelation matrix are modified to equal constants, which can reduce the influence of channel autocorrelation, thereby improving the audio quality of the audio data after dereverberation.

[0011] In a possible implementation, performing dereverberation processing on the frame of audio data based on the second autocorrelation matrix to obtain processed audio data includes:

[0012] When each value on the off-diagonal line of the second autocorrelation matrix is greater than or equal to the first threshold, the step of performing dereverberation processing on the frame of audio data based on the second autocorrelation matrix to obtain processed audio data is performed.

[0013] In a possible implementation, the method further includes:

[0014] If there are values on the off-diagonal line of the second autocorrelation matrix that are smaller than the first threshold, modify the values on the off-diagonal line of the second autocorrelation matrix that are smaller than the first threshold to 0, thereby obtaining a third autocorrelation matrix;

[0015] De-reverberation processing is performed on the frame of audio data based on the third autocorrelation matrix to obtain processed audio data.

[0016] In a possible implementation, the method further includes:

[0017] In a case where the one frame of audio data is the first frame of audio data in the audio data to be processed, an average of cross-correlation results between the first frame of audio data and first frames of audio data collected through other channels is used as the initial variance; the other channels are channels other than the first channel in the electronic device for collecting the sound source signal;

[0018] The dereverberation processing of the frame of audio data based on the second autocorrelation matrix to obtain processed audio data includes:

[0019] De-reverberation processing is performed on the first frame of audio data based on the second autocorrelation matrix and the initial variance to obtain processed audio data.

[0020] In a possible implementation, the method further includes:

[0021] When the frame of audio data is not the first frame of audio data in the audio data to be processed, weighting the first variance and the second variance based on the difference between the first direction angle and the second direction angle to obtain a third variance;

[0022] The first direction angle is the direction angle between the sound source and the position of the first channel within the time period of the previous frame of audio data, and the second direction angle is the direction angle between the sound source and the position of the first channel within the time period of the second frame of audio data before the previous frame of audio data. The first variance is the variance obtained by averaging the cross-correlation results between the previous frame of audio data and the corresponding frames of audio data collected from other channels. The second variance is the variance obtained by deriving the audio data after dereverberation processing based on the previous frame of audio data. The sound source is used to generate the sound source signal.

[0023] The dereverberation processing of the frame of audio data based on the second autocorrelation matrix to obtain processed audio data includes:

[0024] De-reverberation processing is performed on the frame of audio data based on the second autocorrelation matrix and the third difference to obtain processed audio data.

[0025] In a possible implementation, the method further includes:

[0026] In a case where the signal amplitude of the processed audio data is greater than the signal amplitude of the one frame of audio data, the signal amplitude of the processed audio data is modified to the signal amplitude of the one frame of audio data.

[0027] In a second aspect, an embodiment of the present application provides an audio data processing device, comprising a unit for executing the method in the first aspect or any possible implementation of the first aspect.

[0028] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory; the memory is used to store data and computer-executable instructions; and the processor is used to execute the computer-executable instructions stored in the memory so that the method in the first aspect or any possible implementation of the first aspect is executed.

[0029] In a fourth aspect, an embodiment of the present application provides a chip comprising a logic circuit and an interface, wherein the logic circuit and the interface are coupled; the interface is used to input and / or output code instructions, and the logic circuit is used to execute the code instructions so that the method in the first aspect or any possible implementation of the first aspect is executed.

[0030] In the fifth aspect, the present application provides a module device, which includes a communication module, a power module, a storage module and a chip module, wherein the power module is used to provide power to the module device; the storage module is used to store data and instructions; the communication module is used to conduct internal communication within the module device, or for the module device to communicate with external devices; the chip module is used to execute the method in the first aspect or any possible implementation of the first aspect.

[0031] In a sixth aspect, an embodiment of the present application discloses a computer program product, which includes program instructions. When the program instructions are executed by a processor, the method in the first aspect or any possible implementation of the first aspect is executed.

[0032] In the seventh aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a processor, the method in the first aspect or any possible implementation of the first aspect is executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a structural diagram of a dereverberation model provided in an embodiment of the present application;

[0034] Figure 2 This is a flowchart of an audio data processing method provided by an embodiment of the present application;

[0035] Figure 3 is a structural diagram of an electronic device 30 provided in an embodiment of the present application;

[0036] Figure 4 is a structural diagram of an electronic device 40 provided in an embodiment of the present application;

[0037] Figure 5 It is a structural diagram of a module device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and appended claims of the present application, the singular expressions "a", "a kind of", "above", "the" and "this" are intended to also include plural expressions unless there is a clear contrary indication in the context. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations comprising one or more of the listed items. The terms "first" and "second" in the specification, claims and drawings of the present application are used to distinguish different objects, rather than to describe a specific order.

[0039] In everyday life, the sounds audible to the human ear are mechanical waves, and therefore, sound can be understood as sound waves. As sound waves propagate through a propagation medium and reach a receiving end, they are often reflected by obstacles. For example, the receiving end can also be referred to as a sound collection device, such as a microphone.

[0040] For example, when sound waves propagate indoors, they are reflected by obstacles such as walls, ceilings, and floors. With each reflection, a portion of the sound wave is absorbed by the obstacle. In this case, after the sound source ceases to emit sound, the sound wave disappears after multiple reflections and absorptions within the room. Therefore, multiple sound waves continue to mix for a period of time after the sound source ceases to emit sound. This phenomenon of sound persisting even after the sound source ceases to emit sound indoors is called reverberation, and its duration is called reverberation time.

[0041] As another example, in actual environments, voice, as an important information carrier for communication between people and between people and machines, is inevitably subject to interference from reverberation, environmental noise, etc., which greatly reduces the clarity, intelligibility and comfort of the voice signal received by the receiving end, seriously affecting the human ear's auditory perception or the performance of subsequent voice processing.

[0042] It is precisely because of the reverberation phenomenon that the audio signal received by the receiving end includes not only the direct part of the sound source, but also the reflected signal that arrives due to multipath propagation when the sound source passes through the propagation channel, which causes the quality of the audio signal received by the receiving end to deteriorate. Among them, in the embodiment of the present application, the above-mentioned audio signal can be a signal generated or output by any sound source, such as a voice signal of a person speaking, or an audio signal output by an audio output module of an electronic device. It can be understood that since the audio signal received by the receiving end includes the reverberation part, the above-mentioned audio signal can also be called a reverberation audio signal, an audio signal to be processed, a reverberation audio signal to be processed, a signal to be processed, etc. Exemplarily, the data obtained by the receiving end receiving the audio signal can be called audio data.

[0043] In order to offset the sound quality degradation caused by reverberation, dereverberation (or reverberation elimination) technology has emerged. Among them, dereverberation technology can also be called reverberation elimination technology, voice dereverberation technology, etc. Generally, dereverberation technology can be divided into single-channel dereverberation and multi-channel dereverberation technology. Among them:

[0044] Single-channel dereverberation technology uses a single microphone for speech enhancement. This method has been widely used due to its simple model and low cost. However, since single-channel dereverberation technology can only suppress reverberation by utilizing the statistical characteristics of a single signal, the dereverberation effect is limited.

[0045] Multi-channel dereverberation technology uses multiple microphones, or microphone arrays, to capture audio signals, generating multi-channel signals. As the number of input channels increases, multi-channel dereverberation algorithms based on microphone arrays can leverage not only the time-frequency information of a single signal, but also the correlations between multiple signals and their spatial information for speech enhancement. Compared to the limitation of single-channel dereverberation, which can only enhance speech by utilizing the differences between speech and reverberation in the time-frequency domain, the introduction of microphone arrays can overcome the shortcomings of single-channel dereverberation technology.

[0046] Multichannel linear prediction (MCLP) is a method within multichannel dereverberation technology. The MCLP model represents the currently received audio signal as a linear combination of the current clean audio signal and signals received several frames in the past. The goal of dereverberation is to estimate the correlation coefficient and, based on this correlation coefficient, remove the reverberation from the processed audio signal. In other words, the desired speech, namely the direct sound and early reflections, is extracted from the processed audio signal.

[0047] The MCLP algorithm assumes that the sound source signal has a Gaussian distribution with time-varying variance, and the expected speech follows a Gaussian distribution, and the speech signal has correlation in a short time. For ease of understanding, for example, see Figure 1 , Figure 1 This is a structural diagram of a dereverberation model provided in an embodiment of the present application.

[0048] like Figure 1 As shown in the figure, s(t) represents the sound source signal, that is, the pure audio signal without reverberation; h1(z)-h m (z) represents the room impulse response (RIR) of different acoustic channels; v1(t)-v m (t) represents the additive noise in different acoustic channels, x1(t)-x m (t) represents the audio signal to be processed received by different microphones, and the audio signal to be processed includes a reverberation part; taking the m-th microphone as an example, illustratively, the audio signal to be processed x received by the m-th microphone m (t) can be expressed by formula (1):

[0049]

[0050] In formula (1), s(tk) represents the pure audio signal in the time domain; h m (k) indicates that the distance between the sound source signal and the mth microphone is L h The room impulse response of v m(t) represents additive background noise. It is understandable that, for simplicity, the subsequent formulas can be derived without noise. Therefore, the above v m (t) can be equal to 0.

[0051] In MCLP, the audio signal to be processed received by the microphone can be represented by a linear combination of the desired signal (i.e., the direct sound and the early reflection part) and the late reverberation signal. Taking the mth microphone as an example, for example, the audio signal to be processed received by the mth microphone is x m (t) can be expressed by formula (2):

[0052]

[0053] In formula (2), d m (t) represents the expected signal, r m (t) represents the late reverberation signal, and D represents the start time of the late reverberation.

[0054] In the MCLP algorithm, the maximum likelihood estimation method is used to calculate the above correlation coefficient, where the likelihood function can be expressed as formula (3):

[0055]

[0056] In formula (3), T represents the maximum delay of the observed signal; N represents The probability density function of accords with the normal distribution; represents the expected signal; σ 2 (t) is the variance expressed in the time domain, c1 is the correlation coefficient expressed in the time domain, const represents the constant part,

[0057] in, L c represents the order of the regression vector. Finally,

[0058] The above is the representation of the likelihood function in the time domain. In order to simplify the calculation, the likelihood function can be converted to the frequency domain, such as formula (4):

[0059]

[0060] Among them, p(d n,f ) is the probability density function of the complex Gaussian random process, n represents the frame index, f represents the frequency index, ρ f 2 is the variance expressed in the frequency domain, c f is the correlation coefficient expressed in the frequency domain, and const represents the constant part.

[0061] In the above formula (4), in, Among them, L c is the order of the regression vector.

[0062] It can be understood that in formula (4), by θ f When the maximum likelihood function is obtained, the dereverberation is considered to be completed, that is, the audio signal after dereverberation is obtained. However, in reality, it is difficult to obtain a closed solution to the maximum likelihood function in the above formula (4). Therefore, it can be solved by continuously iterating c for each frame of audio data. f and ρ f 2 In the embodiment of the present application, a frame of audio data can be understood as audio data with a duration equal to a threshold. For example, the threshold can be 5 milliseconds, 10 milliseconds, 15 milliseconds, etc., which is not limited in the present application.

[0063] In the process of dereverberation processing of each frame of audio data to be processed, how to improve the dereverberation effect is the direction that technicians in this field are constantly working on. Based on the above problems, an embodiment of the present application provides an audio data processing method. By dereverberation of the audio data to be processed, the audio quality of the dereverberated audio data can be improved. The method provided in the embodiment of the present application can be executed by an electronic device. For example, the above electronic device can be a mobile phone, a tablet, a laptop computer and other devices, which is not limited by the present application.

[0064] For example, see Figure 2 , Figure 2 This is a flow chart of an audio data processing method provided by an embodiment of the present application. Figure 2 As shown, the above method includes:

[0065] 201: Acquire a frame of audio data from audio data to be processed, where the audio data to be processed is audio data obtained by collecting a sound source signal through a first channel.

[0066] In this embodiment, the audio data to be processed is audio data obtained by collecting a sound source signal through a first channel, wherein the first channel can be understood as one of at least two channels, such as audio data to be processed received by a microphone.

[0067] In one possible implementation, the electronic device may include multiple channels for receiving sound source signals, so that the electronic device can directly obtain the above-mentioned audio data to be processed through its own channels. For example, if the electronic device has a 3*3 microphone array, the above-mentioned audio data to be processed can be the audio data collected by one of the microphones.

[0068] In another possible implementation, the electronic device may establish a communication connection with other electronic devices to obtain the audio data to be processed from the other electronic devices.

[0069] 202: Calculate a first autocorrelation matrix based on the frame of audio data.

[0070] In the embodiment of the present application, the electronic device performs dereverberation processing on the audio data to be processed in units of frames. It can be understood from the description of equations (3) and (4) above that the electronic device can calculate the correlation coefficient based on the above-mentioned frame of audio data. During the calculation of the correlation coefficient, an autocorrelation matrix is first calculated. In this step, the above-mentioned first autocorrelation matrix can be understood as the initial autocorrelation matrix, that is, the autocorrelation matrix that has not been modified in step 203.

[0071] 203: When the values on the diagonal of the first autocorrelation matrix are not equal, modify the values on the diagonal of the first autocorrelation matrix to equal constants to obtain a second autocorrelation matrix.

[0072] After obtaining the first autocorrelation matrix, when the values on the diagonal of the first autocorrelation matrix are not equal, the electronic device modifies the values on the diagonal of the first autocorrelation matrix to equal constants, wherein the above constants can be arbitrary constants, such as negative numbers, 0, positive numbers, decimals, and fractions, etc., and this application does not limit this.

[0073] In this step, the unequal values on the diagonal of the first autocorrelation matrix can be understood as the presence of different values on the diagonal of the first autocorrelation matrix. It is understood that if the values on the diagonal of the first autocorrelation matrix are equal constants, the electronic device can perform dereverberation processing on the frame of audio data directly based on the first autocorrelation matrix without modifying the first autocorrelation matrix. In the above process, the methods described in other embodiments of the present application can still be used to optimize the dereverberation effect.

[0074] 204: Perform dereverberation processing on the frame of audio data based on the second autocorrelation matrix to obtain processed audio data.

[0075] In this step, the electronic device uses the modified second autocorrelation matrix to perform dereverberation processing on the frame of audio data. Exemplarily, the electronic device can calculate a correlation coefficient based on the second autocorrelation matrix, and then perform dereverberation processing on the frame of audio data according to the correlation coefficient.

[0076] In an embodiment of the present application, for the above-mentioned frame of audio data, when the values on the diagonal of the calculated first autocorrelation matrix are not equal, the values on the diagonal of the first autocorrelation matrix are modified to equal constants. Since the diagonal in the autocorrelation matrix reflects the autocorrelation characteristics, the different values on the diagonal of the autocorrelation matrix mean that the autocorrelation matrix includes the influence of the first channel itself. In the process of dereverberation processing of the above-mentioned frame of audio data, the values on the diagonal of the first autocorrelation matrix are modified to equal constants, which can reduce the influence of channel autocorrelation, thereby improving the audio quality of the audio data after dereverberation.

[0077] In some embodiments, Figure 2 In the method shown, step 204 includes:

[0078] 2401: When each non-diagonal value in the second autocorrelation matrix is greater than or equal to a first threshold, perform the step of performing dereverberation processing on the frame of audio data based on the second autocorrelation matrix to obtain processed audio data.

[0079] It can be understood that the smaller values on the off-diagonal lines in the autocorrelation matrix reflect small noise interference. In this embodiment, when each value on the off-diagonal lines in the second autocorrelation matrix is greater than or equal to the first threshold, dereverberation processing is performed on the above-mentioned frame of audio data based on the second autocorrelation matrix. This can ensure that there is no small noise interference during the dereverberation processing, while achieving the purpose of reducing the amount of calculation.

[0080] In the embodiment of the present application, the first threshold value can be understood as a number that is much smaller than the signal amplitude of the input audio data (e.g., the one frame of audio data) and very close to 0. For example, if the amplitude autocorrelation of the input audio data is 0.5, the first threshold value can be a number that is less than 1% of the amplitude autocorrelation, such as 0.01, 0.02, etc.

[0081] In some other embodiments, the embodiment in which step 2401 is located further includes:

[0082] 2042: If there are values on the off-diagonal line of the second autocorrelation matrix that are smaller than the first threshold, modify the values on the off-diagonal line of the second autocorrelation matrix that are smaller than the first threshold to 0, thereby obtaining a third autocorrelation matrix;

[0083] 2043: Perform dereverberation processing on the frame of audio data based on the third autocorrelation matrix to obtain processed audio data.

[0084] In this embodiment, when there are values on the off-diagonal line of the second autocorrelation matrix that are smaller than the above-mentioned first threshold, the numbers on the off-diagonal line of the second autocorrelation matrix that are smaller than the above-mentioned first threshold are modified to 0 to obtain a third autocorrelation matrix. De-reverberation processing is performed based on the third autocorrelation matrix. This can reduce noise interference when there is noise disturbance in the autocorrelation matrix, thereby reducing the amount of calculation.

[0085] In the embodiment of the present application, in addition to optimizing the autocorrelation matrix, the variance of the dereverberation process can also be optimized. In some embodiments, Figure 2 The illustrated method further includes:

[0086] 205: When the frame of audio data is the first frame of audio data in the audio data to be processed, taking an average of cross-correlation results between the first frame of audio data and first frames of audio data collected by other channels as an initial variance; the other channels are channels other than the first channel in the electronic device for collecting the sound source signal;

[0087] Step 204 includes:

[0088] 2044: Perform dereverberation processing on the first frame of audio data based on the second autocorrelation matrix and the initial variance to obtain processed audio data.

[0089] In this embodiment, the one frame of audio data being the first frame of audio data in the audio data to be processed can be understood as the first frame of data or the initial frame of data in the audio data to be processed. It is understood that before performing dereverberation processing on the first frame of audio data, the variance needs to be initialized.

[0090] In an embodiment of the present application, the above-mentioned other channels are channels other than the above-mentioned first channel in the electronic device for collecting the above-mentioned sound source signals, wherein the electronic device may be the same device as the electronic device of the executing subject, or may be a different device. In the case of the same device, the electronic device that executes the method in this embodiment includes multiple channels for receiving the same sound source signal; in the case of not being the same device, other electronic devices that establish a communication connection with the electronic device that executes the method in this embodiment may include multiple channels for receiving the same sound source signal, that is, the audio data collection is completed by other electronic devices, and the dereverberation processing is completed by the electronic device in the embodiment of the present application. In this embodiment, for the first frame of audio data, the average value of the cross-correlation results between the first frame of audio data and the first frame of audio data collected by other channels is used as the initial variance.

[0091] Taking channel A as an example, the first frame of audio data and the first frame of audio data collected by channel A undergo Fourier transform to obtain respective spectral sequences. Cross-correlating the spectral sequences yields a cross-correlation sequence, referred to as a cross-correlation sequence. The cross-correlation sequence can be understood as the cross-correlation result, and the values at different positions in the cross-correlation sequence represent the cross-correlation coefficients between the first channel and channel A at different frequencies.

[0092] Therefore, after cross-correlating the first frame of audio data collected on the first channel with the first frames of audio data collected on the other channels, multiple cross-correlation sequences are obtained. In this embodiment, the average value of the cross-correlation results can be understood as a sequence obtained by averaging the cross-correlation coefficients corresponding to the same frequency points in the multiple cross-correlation sequences.

[0093] After obtaining the initial variance in the above manner, in this embodiment, dereverberation processing is performed on the first frame of audio data based on the initial variance and the second autocorrelation matrix. Optionally, when the second autocorrelation matrix is modified to a third autocorrelation matrix, the first frame of audio data can also be processed based on the initial variance and the third autocorrelation matrix.

[0094] It is understandable that in other solutions, the variance also needs to be initialized before dereverberation is performed on the first frame of audio data. However, in these solutions, the signal power of the first frame of audio data is used as the initialization variance, ignoring the spatial coupling between channels. In this embodiment, the variance is initialized using the cross-correlation results between the first channel and the first frame of audio data in other channels. That is, a time-invariant full correlation matrix is used to express the short-time average coupling between channels. This can enhance the spatial coupling characteristics between channels and improve the audio quality after dereverberation processing.

[0095] In the above, the variance is initialized based on the first frame of audio data in different channels. For other frames after the first frame of audio data, the variance can be obtained based on the dereverberation processing result of the previous frame and the directional angle between the sound source position and the channel in the electronic device, thereby optimizing the dereverberation processing performance when the sound source position changes.

[0096] In some embodiments, the above Figure 2 The illustrated method further includes:

[0097] 206: When the frame of audio data is not the first frame of audio data in the audio data to be processed, weight the first variance and the second variance based on the difference between the first direction angle and the second direction angle to obtain a third variance; wherein the first direction angle is the direction angle between the sound source and the position of the first channel within a time period of the previous frame of audio data, and the second direction angle is the direction angle between the sound source and the position of the first channel within a time period of the second previous frame of audio data; the first variance is the variance obtained by averaging cross-correlation results between the previous frame of audio data and frames of audio data collected corresponding to other channels, and the second variance is the variance obtained by deriving audio data from the previous frame of audio data after dereverberation processing; the sound source is used to generate the sound source signal;

[0098] Step 204 includes:

[0099] 2045: Perform dereverberation processing on the frame of audio data based on the second autocorrelation matrix and the third difference to obtain processed audio data.

[0100] In this embodiment, the above-mentioned one frame of audio data is not the first frame of audio data in the above-mentioned audio data to be processed. It can be understood that the above-mentioned one frame of audio data is a frame of audio data after the first frame of audio data in the above-mentioned audio data to be processed, for example, it can be the second frame of audio data, the fifth frame of audio data, etc.

[0101] In the embodiment of the present application, the sound source is used to generate the above-mentioned sound source signal, and the above-mentioned direction angle can be understood as the direction of arrival (DOA) angle, which is used to indicate the relative position between the sound source position and the receiving end.

[0102] It's understandable that channels are located in different locations within an electronic device. Consequently, different channels are located in different locations. Furthermore, due to the different spatial locations of the channels, different channels receive the aforementioned sound source signal (i.e., the same sound source signal) at different times. Consequently, different channels receive the sound source signal with different time delays, and the direction of arrival (DOA) can be calculated based on these time delay differences.

[0103] In this embodiment, the first direction angle can be understood as the arrival direction angle obtained based on the time delay difference between the first channel receiving the above-mentioned frame of audio data and the other channels receiving the corresponding audio data, wherein the above-mentioned corresponding audio data can be understood as a frame of audio data with the same position as the above-mentioned frame of audio data. For example, if the above-mentioned frame of audio data is the 10th frame of audio data in the audio data to be processed, then the above-mentioned corresponding audio data is the 10th frame of audio data of the audio data to be processed received by other channels.

[0104] It's understandable that, because a frame of audio data is relatively short, the relative position between the sound source and the first channel can be assumed to remain unchanged during that time. Therefore, the first direction angle can be understood as the direction angle between the sound source and the first channel during the time period of the frame of audio data. Similarly, the second direction angle can be understood as the direction angle between the sound source and the first channel during the time period of the previous frame of audio data. Therefore, the difference between the first and second direction angles represents the change in the sound source's position relative to the receiving channel.

[0105] It is understandable that an initial variance is required for dereverberation of each frame of audio data. In other solutions, when dereverberation is performed on a frame of audio data (except the first frame of audio data), the initial variance is calculated based on the audio data obtained after dereverberation of the previous frame of audio data. For example, when dereverberation is performed on the fifth frame of audio data, the initial variance of the fifth frame of audio data may be the square of the signal amplitude of the audio data obtained after dereverberation of the fourth frame of audio data.

[0106] In this embodiment, the first variance is a variance obtained by averaging the cross-correlation results between the previous frame of audio data of the frame of audio data and the frames of audio data collected corresponding to other channels, and the second variance is a variance obtained by deriving the audio data from the previous frame of audio data after dereverberation processing.

[0107] Exemplarily, the frame of audio data is the second frame of audio data. The first variance may be the initial variance of the first frame of audio data determined in step 205, referred to as the first frame variance. The second variance may be the square of the signal amplitude of the audio data obtained after dereverberation processing is performed on the first frame of audio data, referred to as the first frame dereverberation variance.

[0108] In particular, for the second frame of audio data, the first direction angle and the second direction angle can be understood as the direction angle between the sound source and the position of the first channel within the time period of the first frame of audio data. Therefore, the difference between the first direction angle and the second direction angle is 0.

[0109] Finally, the first frame variance and the first frame dereverberation variance are weighted based on the above difference to obtain the initial variance of the second frame of audio data, i.e., the above third variance. To distinguish it from the variance in the subsequent third frame of audio data, the initial variance of the second frame of audio data is referred to as the second frame initial variance.

[0110] As another example, for the third frame of audio data, the first variance may be the average value of the cross-correlation results between the second frame of audio data and the corresponding frames of audio data collected by other channels, referred to as the second frame variance. The specific calculation method is similar to that of step 205 above and will not be repeated here. In this embodiment, the corresponding frame of audio data collected may be understood as a frame of audio data at the same position as the previous frame of audio data. For example, if the above-mentioned frame of audio data is the third frame of audio data to be processed, then the corresponding frame of audio data collected is the second frame of audio data to be processed received by other channels. The second variance may be the square of the signal amplitude of the audio data obtained after dereverberation processing of the second frame of audio data, referred to as the second frame dereverberation variance.

[0111] In this embodiment of the present application, the second frame of audio data preceding the aforementioned frame of audio data can be understood as the second frame of audio data preceding the aforementioned frame of audio data. For the third frame of audio data, the first direction angle can be understood as the direction angle between the sound source and the position of the aforementioned first channel during the time period of the second frame of audio data, and the second direction angle can be understood as the direction angle between the sound source and the position of the aforementioned first channel during the time period of the first frame of audio data. Therefore, the difference between the aforementioned first direction angle and the second direction angle may be 0 or may not be 0.

[0112] Finally, the second frame variance and the second frame dereverberation variance are weighted based on the above difference to obtain the initial variance of the third frame audio data.

[0113] In this embodiment, there can be multiple weighting methods in step 206. The sine or cosine of the difference between the first direction angle and the second direction angle can be calculated to obtain a number in the range of [0,1] (called a transformation number), and then the first variance and the second variance mentioned above can be weighted using the transformation number and 1 minus the change number as coefficients to obtain a third variance.

[0114] In some embodiments, the weighting coefficient of the first variance is equal to 1 minus a cosine value, where the cosine value is the cosine value of the difference between the first direction angle and the second direction angle, and the weighting coefficient of the second variance is equal to the cosine value. In this embodiment, when the first direction angle and the second direction angle are the same, that is, the relative position between the sound source and the first channel remains unchanged, the third variance is equal to the second variance; when the angle between the first direction angle and the second direction angle is 90 degrees, the third variance is equal to the first variance.

[0115] It is understandable that the initial variance of each subsequent frame of audio data can be obtained in a similar manner as the second frame of audio data and the third frame of audio data described above. After iteration of multiple frames of audio data, the dereverberation effect will become better and better.

[0116] In the embodiment of the present application, for the first frame of audio data in the audio data to be processed, the initial variance is obtained using the method described in step 205. For the initial variances of the other frames of audio data, the initial variances are obtained based on the variance of the previous frame of audio data and the first and second directional angles. The variance is modified using the directional angles corresponding to the historical audio data. The variance obtained in this way is time-varying and can better adapt to the time-varying situation where the relative position between the sound source position and the receiving channel changes, thereby improving the audio quality obtained by the dereverberation processing under time-varying conditions.

[0117] It is understandable that the input audio data includes reverberation and noise interference. Under normal circumstances, the signal amplitude of the audio data after dereverberation processing should be smaller than the signal amplitude of the input audio data. However, it is not ruled out that the dereverberation processing may make mistakes, that is, distortion. Therefore, in some embodiments, the above Figure 2 The illustrated method further includes:

[0118] 207 : If the signal amplitude of the processed audio data is greater than the signal amplitude of the one-frame audio data, modify the signal amplitude of the processed audio data to the signal amplitude of the one-frame audio data.

[0119] In this embodiment, when the signal amplitude of the processed audio data is greater than the signal amplitude of the above-mentioned frame of audio data, the signal amplitude of the processed audio data is modified to the signal amplitude of the above-mentioned frame of audio data, that is, the signal amplitude of the audio data before processing.

[0120] It can be understood that, from the relevant embodiments of step 207 above, for two adjacent frames of audio data, the initial variance of the latter frame of audio data is obtained based on the audio data after dereverberation processing of the previous frame of audio data. By limiting the signal amplitude of the processed audio data, the robustness of the algorithm can be improved and the impact of update errors caused by transient noise can be reduced.

[0121] It should be noted that in the embodiments of the present application, the numbers before the steps are made to facilitate the description and understanding of the scheme and should not be understood as limiting the execution order of the steps.

[0122] In order to more clearly understand the audio data processing method provided in the embodiment of the present application, the dereverberation processing process of the audio data is introduced below with reference to the formula.

[0123] For example, the channels in an electronic device are microphones, meaning the electronic device includes a microphone array, each microphone being used to collect audio signals. It is understood that the audio signals received by the electronic device via the microphones are analog audio signals. After converting the analog audio signals into electrical signals, the electrical signals are sampled, quantized, and encoded to obtain digital audio signals. The digital audio signals are then Fourier transformed from the time domain to the frequency domain to obtain the audio data to be processed. It is understood that each microphone can obtain its own audio data to be processed using the aforementioned methods.

[0124] Taking the dereverberation processing of the audio data to be processed (audio data to be processed A) obtained by a certain microphone (referred to as microphone A) as an example, the electronic device obtains the first frame of audio data to be processed in the audio data to be processed A and performs the following steps:

[0125] (1) Initialize the variance p f 2

[0126] In this step, the audio data received by microphone A and the audio data received by other microphones are initialized with a variance. For example, the variance p f 2 It can be expressed by formula (5):

[0127]

[0128] Where m represents the number of channels, i.e., the number of microphones. n represents the frame index, and f represents the frequency index. x represents the audio data to be processed received by the microphone, and the superscript of x represents the microphone index. For example, a superscript of 1 represents the audio data to be processed received by microphone A, and other superscripts represent the audio data to be processed received by other microphones. conj represents the conjugate of the complex number, and ε represents the preset minimum value. If the value calculated based on the audio data to be processed is less than ε, the variance is taken as ε.

[0129] It can be understood that in the MCLP algorithm, it is assumed that the sound source signal is a Gaussian distribution with time-varying variance, and the speech is expected to obey the Gaussian distribution. When using the MCLP algorithm for dereverberation, the input signal power of the audio data to be processed received by microphone A is used as the initial variance. However, using only the input signal power for variance initialization ignores the spatial coupling between channels.

[0130] To enhance the spatial coupling characteristics between channels, this application uses the product of the conjugate transpose of the unprocessed audio data received by microphone A and the unprocessed audio data received by other microphones as the initial variance, that is, the average of the cross-correlation results of the unprocessed audio data received by microphone A and the unprocessed audio data received by other microphones as the initial variance. The variance obtained in this way is time-invariant and can be used to express the short-time average coupling of the channels. Because this application takes into account the spatial coupling between channels, the dereverberation performance is better.

[0131] (2) Modify the autocorrelation matrix φ

[0132] Based on the above formula (4), the expression (6) of the correlation coefficient can be obtained:

[0133]

[0134] Among them, n represents the frame index, f represents the frequency index, and the autocorrelation matrix

[0135] In the embodiment of the present application, during the process of calculating the correlation coefficient based on the autocorrelation matrix φ, the autocorrelation matrix is modified so that the values on the diagonal of the autocorrelation matrix are equal and constant, thereby reducing the influence of channel autocorrelation. The autocorrelation matrix before modification can be understood as the first autocorrelation matrix described above, and the autocorrelation matrix after modification can be understood as the second autocorrelation matrix described above.

[0136] In addition, by utilizing the sparse characteristic, the values in the autocorrelation matrix that are less than the first threshold are modified to 0. The obtained autocorrelation matrix can be understood as the third autocorrelation matrix mentioned above, thereby reducing the disturbance caused by noise and the amount of calculation.

[0137] (3) Obtaining dedeverberation audio data

[0138] For example, the dereverberated audio data d (n,f) It can be understood that the audio data after the above processing can be expressed by formula (7):

[0139]

[0140] Where n represents the frame index and f represents the frequency index. The first frame of audio data is processed to obtain After that, the dereverberation processing of the first frame of audio data can be considered completed.

[0141] (4) Disturbance detection

[0142] In the embodiment of the present application, the audio data after dereverberation is Adding offset detection, the offset detection can be expressed by the following formula (8):

[0143]

[0144] It is understandable that when the audio data is clean or reverberant, the correlation coefficient calculated by Equation (6) is not zero due to the short-term correlation and time delay of the speech signal. Currently, the output signal after dereverberation processing using the MCLP algorithm may be distorted, that is, the amplitude of the output signal may be greater than the amplitude of the input signal. When the output signal amplitude is greater than the input signal amplitude, it indicates that the MCLP algorithm generates redundant information at that frequency point.

[0145] In the embodiment of the present application, That is, when the output signal amplitude is less than or equal to the input signal amplitude, the output signal is not modified, that is, the output is calculated by formula (7) In satisfaction That is, when the output signal amplitude is greater than the input signal amplitude, the output signal amplitude is modified to the input signal amplitude, that is, the output signal amplitude is multiplied by the ratio of the input signal amplitude to the output signal amplitude, that is, This reduces redundant information.

[0146] (5) Update variance

[0147] For example, the variance can be updated using formula (9):

[0148]

[0149] In formula (9), Represents the updated variance, which is used for dereverberation processing of the next frame of audio data of the current frame. The variance obtained by using the input signal corresponding to the current frame may be the average value of the cross-correlation results between a frame of audio data received by microphone A and a frame of audio data received by other microphones. Indicates the current frame obtained after dereverberation processing The calculated variance can be The square of the signal amplitude is taken. Optionally, the variance The calculation sets the minimum value ε, when When ε is less than ε, Take the minimum value of ε.

[0150] In formula (9), a(n)=cos(θ(n)-θ(n-1)), where θ(n) represents the direction of arrival angle estimated in the current frame, and θ(n-1) represents the direction of arrival angle estimated in the frame before the current frame.

[0151] Exemplarily, when the current frame is the first frame of audio data received by microphone A, and the dereverberation processing of the first frame of audio data is completed through the above steps (1) to (4), the variance is updated according to formula (9). In particular, for the first frame of audio data, θ(n) = θ(n-1) is preset. Therefore, after the dereverberation processing of the first frame of audio data, the updated variance is obtained. For ease of understanding, the variance obtained by updating based on the first frame of audio data is referred to as a first frame dereverberation variance. The first frame dereverberation variance is used for dereverberation processing of the second frame of audio data.

[0152] For the second frame of audio data received by microphone A, the dereverberation variance of the first frame is used as the initial variance for dereverberation processing. The dereverberation processing process is similar to that of the first frame of audio data, namely, steps (2) to (4) above. After the dereverberation processing of the second frame of audio data is completed, the variance is updated according to the above formula (9).

[0153] It can be understood that, for the second frame of audio data, θ(n) represents the estimated direction of arrival angle of the second frame of audio data, and θ(n-1) represents the estimated direction of arrival angle of the first frame of audio data; is the average value of the cross-correlation results between the second frame of audio data received by microphone A and the second frame of audio data received by other microphones; The second frame of audio data is obtained after de-reverberation processing Therefore, the variance obtained by updating the second frame of audio data is the result of weighted summation. For ease of understanding, the variance obtained based on the second frame of audio data is called the second frame dereverberation variance, which is used for dereverberation processing of the third frame of audio data.

[0154] It is understandable that the dereverberation processing and variance update of subsequent frames are similar to those of the second frame of audio data, and will not be repeated here. In this way, through continuous iteration, the dereverberation processing effect of subsequent frames will become better and better.

[0155] It can be understood that the above process updates the variance from the perspective of the current frame that has completed the dereverberation process. If we look at it from the perspective of the next frame of audio data, that is, the next frame of unprocessed audio data is the current frame, then, It can be understood as the variance obtained by using the input signal corresponding to the previous frame of the current frame. Therefore, the above It can be understood as the first variance mentioned above. Similarly, in formula (9) It can be understood as the previous frame of the current frame after dereverberation processing. The calculated variance is, therefore, It can be understood as the second variance mentioned above.

[0156] Similarly, the above θ(n) can be understood as the estimated direction of arrival angle of the frame before the current frame, so θ(n) can be understood as the first direction angle mentioned above; the above θ(n-1) can be understood as the estimated direction of arrival angle of the second frame before the current frame, so θ(n-1) can be understood as the second direction angle mentioned above. Finally, based on the above above The results obtained by the above θ(n) and the above θ(n-1) can be understood as the above third difference, and the dereverberation processing is performed on the current frame based on the third difference.

[0157] The method of the embodiment of the present application is described in detail above, and the device provided by the embodiment of the present application is described below.

[0158] For example, see Figure 3 , Figure 3 Schematic diagram of the structure of an electronic device 30 provided in an embodiment of the present application. Figure 3 As shown, the electronic device 30 includes an acquisition unit 301, a processing unit 302, and a modification unit 303. Optionally, it may also include a determination unit 304, which is described in detail as follows:

[0159] An acquiring unit 301 is configured to acquire a frame of audio data from the audio data to be processed, where the audio data to be processed is audio data acquired by collecting a sound source signal through a first channel.

[0160] The processing unit 302 is configured to calculate a first autocorrelation matrix based on the frame of audio data.

[0161] a modifying unit 303 configured to modify the values on the diagonal of the first autocorrelation matrix to equal constants when the values on the diagonal of the first autocorrelation matrix are not equal, thereby obtaining a second autocorrelation matrix;

[0162] The processing unit 302 is further configured to perform dereverberation processing on the frame of audio data based on the second autocorrelation matrix to obtain processed audio data.

[0163] In one possible implementation, the processing unit 302 is specifically configured to, when each value on a non-diagonal line in the second autocorrelation matrix is greater than or equal to a first threshold, perform the step of performing dereverberation processing on the frame of audio data based on the second autocorrelation matrix to obtain processed audio data.

[0164] In a possible implementation, the modifying unit 303 is further configured to, when there are values on the off-diagonal line of the second autocorrelation matrix that are smaller than the first threshold, modify the values on the off-diagonal line of the second autocorrelation matrix that are smaller than the first threshold to 0, thereby obtaining a third autocorrelation matrix;

[0165] The processing unit 302 is further configured to perform dereverberation processing on the frame of audio data based on the third autocorrelation matrix to obtain processed audio data.

[0166] In one possible implementation, the apparatus further includes a determining unit 304 configured to, when the frame of audio data is the first frame of audio data in the audio data to be processed, use an average of cross-correlation results between the first frame of audio data and first frames of audio data collected through other channels as the initial variance; the other channels being channels other than the first channel in the electronic device for collecting the sound source signal;

[0167] The processing unit 302 is specifically configured to perform dereverberation processing on the first frame of audio data based on the second autocorrelation matrix and the initial variance to obtain processed audio data.

[0168] In a possible implementation, the processing unit 302 is further configured to, when the frame of audio data is not the first frame of audio data in the audio data to be processed, weight the first variance and the second variance based on the difference between the first direction angle and the second direction angle to obtain a third variance; wherein the first direction angle is the direction angle between the sound source and the position of the first channel within a time period of the previous frame of audio data of the frame of audio data, and the second direction angle is the direction angle between the sound source and the position of the first channel within a time period of the second frame of audio data before the frame of audio data; the first variance is the variance obtained by averaging the cross-correlation results between the previous frame of audio data of the frame of audio data and the frames of audio data collected corresponding to other channels, and the second variance is the variance obtained by deriving the audio data after dereverberation processing based on the previous frame of audio data of the frame of audio data; the sound source is configured to generate the sound source signal;

[0169] The processing unit 302 is specifically configured to perform dereverberation processing on the frame of audio data based on the second autocorrelation matrix and the third difference to obtain processed audio data.

[0170] In a possible implementation, the modification unit 303 is further configured to modify the signal amplitude of the processed audio data to the signal amplitude of the frame of audio data when the signal amplitude of the processed audio data is greater than the signal amplitude of the frame of audio data.

[0171] See also Figure 4 , Figure 4 Schematic diagram of the structure of an electronic device 40 provided in an embodiment of the present application. Figure 4As shown, the electronic device 40 includes a memory 401 and a processor 402. Optionally, it may further include a communication interface 403 and a bus 404, wherein the memory 401, the processor 402 and the communication interface 403 are communicatively connected to each other via the bus 404.

[0172] Memory 401 is used to provide storage space for storing data such as an operating system and computer programs. Memory 401 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM).

[0173] The processor 402 is a module that performs arithmetic and logical operations, and may be one or a combination of multiple processing modules such as a central processing unit (CPU), a graphics processing unit (GPU), or a microprocessor unit (MPU).

[0174] The memory 401 stores a computer program, and the processor 402 calls the computer program stored in the memory 401 to execute the above-mentioned image registration method. For example, when the above-mentioned electronic device 40 is the above-mentioned electronic device 30, the content obtained by the above-mentioned acquisition unit 301 can be implemented by the communication interface 403, and the steps performed by the above-mentioned processing unit 302, modification unit 303, and determination unit 304 can be implemented by the processor 402.

[0175] An embodiment of the present application also provides a chip. The chip includes a processor and a memory. The processor may be one or more, and the memory may be one or more. The processor reads instructions and data stored in the memory to execute the above-described method and the steps performed in the related embodiments. Of course, the chip may also lack a memory.

[0176] For example, see Figure 5 , Figure 5: is a structural diagram of a module device provided in an embodiment of the present application. The module device 50 can execute the relevant steps of the electronic device in the aforementioned method embodiment. The module device 50 includes: a communication module 501, a power module 502, a storage module 503 and a chip module 504. Among them, the power module 502 is used to provide power to the module device; the storage module 503 is used to store data and instructions; the communication module 501 is used for internal communication within the module device, or for the module device to communicate with external devices, and is used to execute the steps executed by the above-mentioned acquisition unit 301; the chip module 504 can execute the steps executed by the above-mentioned processing unit 302, modification unit 303 and determination unit 304.

[0177] The present application also provides a computer-readable storage medium, in which computer code is stored. When the computer code is run on a computer, the computer executes the method of the above embodiment.

[0178] The present application also provides a computer program product, which includes computer code or computer program. When the computer code or computer program runs on a computer, the method in the above embodiment is executed.

[0179] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the above claims.

Claims

1. A method for processing audio data, characterized in that: The method comprises: Acquire a frame of audio data from the audio data to be processed, where the audio data to be processed is audio data obtained by collecting a sound source signal through a first channel; Calculating a first autocorrelation matrix based on the one frame of audio data; In a case where the values on the diagonal of the first autocorrelation matrix are not equal, modifying the values on the diagonal of the first autocorrelation matrix to equal constants to obtain a second autocorrelation matrix; De-reverberation processing is performed on the frame of audio data based on the second autocorrelation matrix to obtain processed audio data.

2. The method according to claim 1, characterized in that The performing dereverberation processing on the frame of audio data based on the second autocorrelation matrix to obtain processed audio data includes: When each value on the off-diagonal line of the second autocorrelation matrix is greater than or equal to the first threshold, the step of performing dereverberation processing on the frame of audio data based on the second autocorrelation matrix to obtain processed audio data is performed.

3. The method according to claim 2, characterized in that The method further comprises: When there are values on the off-diagonal line of the second autocorrelation matrix that are smaller than the first threshold, modify the values on the off-diagonal line of the second autocorrelation matrix that are smaller than the first threshold to 0, to obtain a third autocorrelation matrix; De-reverberation processing is performed on the frame of audio data based on the third autocorrelation matrix to obtain processed audio data.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: When the frame of audio data is the first frame of audio data in the audio data to be processed, an average value of cross-correlation results between the first frame of audio data and first frames of audio data collected through other channels is used as the initial variance; the other channels are channels other than the first channel in the electronic device for collecting the sound source signal; The performing dereverberation processing on the frame of audio data based on the second autocorrelation matrix to obtain processed audio data includes: De-reverberation processing is performed on the first frame of audio data based on the second autocorrelation matrix and the initial variance to obtain processed audio data.

5. The method according to claim 4, characterized in that The method further comprises: When the frame of audio data is not the first frame of audio data in the audio data to be processed, weighting the first variance and the second variance based on a difference between the first direction angle and the second direction angle to obtain a third variance; The first direction angle is the direction angle between the sound source and the position of the first channel within the time period of the previous frame of audio data of the frame of audio data, and the second direction angle is the direction angle between the sound source and the position of the first channel within the time period of the second frame of audio data before the frame of audio data; the first variance is the variance obtained by averaging the cross-correlation results between the previous frame of audio data of the frame of audio data and the frames of audio data collected corresponding to other channels, and the second variance is the variance obtained by deriving the audio data after the previous frame of audio data of the frame of audio data is derives from the reverberation process; the sound source is used to generate the sound source signal; The performing dereverberation processing on the frame of audio data based on the second autocorrelation matrix to obtain processed audio data includes: De-reverberation processing is performed on the frame of audio data based on the second autocorrelation matrix and the third difference to obtain processed audio data.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: In a case where the signal amplitude of the processed audio data is greater than the signal amplitude of the one-frame audio data, the signal amplitude of the processed audio data is modified to the signal amplitude of the one-frame audio data.

7. An electronic device, characterized in that: The method comprises a unit for executing the method according to any one of claims 1 to 6.

8. An electronic device, characterized in that: include: processor and memory; The memory is used to store data and computer execution instructions; The processor is configured to execute the computer-executable instructions stored in the memory, so that the method according to any one of claims 1 to 6 is performed.

9. A chip, characterized in that: comprising a logic circuit and an interface, wherein the logic circuit and the interface are coupled; The interface is used to input and / or output code instructions, and the logic circuit is used to execute the code instructions so that the method according to any one of claims 1 to 6 is executed.

10. A module device, characterized in that: The module device includes a communication module, a power module, a storage module and a chip module, wherein the power module is used to provide power to the module device; the storage module is used to store data and instructions; the communication module is used for internal communication within the module device, or for the module device to communicate with external devices; the chip module is used to execute the method described in any one of claims 1-6.

11. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the method according to any one of claims 1 to 6 is executed.

Images