An audio processing method based on active matrix

Abstract

The present invention provides an audio processing method based on an active matrix, which belongs to the field of speech processing technology, and includes: collecting a first audio signal based on all audio input devices, and constructing an active matrix based on all the collected first audio signals; analyzing the active matrix, and determining a first target signal and a first non-target signal of all the first audio signals; enhancing the first target signal to determine a second target signal, and suppressing the first non-target signal to determine a second non-target signal; and determining a second audio signal based on the second target signal and the second non-target signal. The method can significantly improve the quality and signal-to-noise ratio of audio signals, improve audio processing effects, improve the accuracy and anti-interference ability of target sound source separation, and realize a dynamic combination of target signal enhancement and interference signal suppression, and is more suitable for high-quality audio processing applications in complex acoustic environments.

Description

Technical Field

[0001] The present invention relates to the technical field of speech processing, and in particular to an audio processing method based on an active matrix. Background Art

[0002] Audio processing is widely used in conference systems, voice communications, intelligent voice interaction and other scenarios. In recent years, audio signal processing technology based on matrix operations has gradually attracted attention. By constructing a matrix containing information collected by multiple devices, the audio signal can be described in all aspects. However, existing technologies usually lack dynamic feedback optimization for target signal enhancement and interference signal suppression, making it difficult to further improve the signal-to-noise ratio. The computational cost is high when processing complex audio signals in real time, and it is difficult to cope with complex dynamic environments.

[0003] Therefore, the present invention provides an audio processing method based on active matrix. Summary of the invention

[0004] The present invention provides an audio processing method based on an active matrix. An active matrix is ​​constructed by collecting all first audio signals, analyzing the active matrix, determining first target signals and first non-target signals of all first audio signals, determining second target signals and second non-target signals, and determining second audio signals. The method can significantly improve the quality and signal-to-noise ratio of audio signals, improve audio processing effects, improve the accuracy of target sound source separation and anti-interference capability, realize a dynamic combination of target signal enhancement and interference signal suppression, and is more suitable for high-quality audio processing applications in complex acoustic environments.

[0005] The present invention provides an active matrix-based audio processing method, comprising:

[0006] 101: Collecting first audio signals based on all audio input devices, and constructing an active matrix based on all collected first audio signals;

[0007] 102: Analyze the active matrix to determine first target signals and first non-target signals of all first audio signals;

[0008] 103: performing signal enhancement on the first target signal to determine a second target signal, and performing signal suppression on the first non-target signal to determine a second non-target signal;

[0009] 104: Determine a second audio signal based on the second target signal and the second non-target signal.

[0010] The method of collecting first audio signals based on all audio input devices and constructing an active matrix based on all collected first audio signals includes:

[0011] Determine the three-dimensional coordinates and device label of each audio input device, and determine the audio input array based on the three-dimensional coordinates and device labels of all audio input devices;

[0012] Collecting and preprocessing the first audio sub-signal of each audio input device, and determining the first audio signal based on the preprocessed first audio sub-signals of all audio input devices collected at the same time point;

[0013] Determine a sampling frequency, and determine a time step vector of all first audio sub-signals in the first audio signal based on the sampling frequency, wherein the time step vector includes a plurality of time points;

[0014] Determine a time domain vector of an audio input device corresponding to each first audio sub-signal based on a time domain signal of each first audio sub-signal in the first audio signal at each time point in the time step vector;

[0015] Determine a frame length and a window function of a short-time Fourier transform, perform a short-time Fourier transform on a time domain vector of an audio input device corresponding to each first audio sub-signal based on the frame length and the window function, and determine a complex spectrum vector of the audio input device corresponding to each first audio sub-signal;

[0016] Constructing an active matrix A based on the time step vector, the complex spectrum vectors of the audio input devices corresponding to all the first audio sub-signals, and the device labels of all the audio input devices; in, 1, N1 represents the number of device tags, , N2 represents the number of time points in the time step vector, , N3 represents the number of frequency dimensions, Represents the complex spectrum value of the audio input device with the mth device tag at the tth time point and the fth frequency dimension, The complex spectrum vectors representing the audio input devices of the mth device label.

[0017] According to an active matrix-based audio processing method provided by the present invention, the active matrix is ​​analyzed to determine the first target signal and the first non-target signal of all first audio signals, including:

[0018] Directional filtering is performed on the active matrix A based on the audio input array to determine the beam target signal matrix;

[0019] Decomposing the active matrix A and optimizing the mask to determine the mask target signal matrix and the mask non-target signal matrix;

[0020] Determine a comprehensive target signal matrix based on the beam target signal matrix and the mask target signal matrix;

[0021] in, represents the comprehensive target signal matrix, represents the beam target signal matrix, represents the mask target signal matrix, Indicates that the fth frequency dimension belongs to the low-frequency dimension, Indicates that the fth frequency dimension belongs to the high-frequency dimension;

[0022] Performing an inverse short-time Fourier transform on the comprehensive target signal matrix to determine first target signals of all first audio signals;

[0023] An inverse short-time Fourier transform is performed on the masked non-target signal to determine first non-target signals of all first audio signals.

[0024] According to an active matrix-based audio processing method provided by the present invention, directional filtering is performed on an active matrix A based on an audio input array to determine a beam target signal, including:

[0025] Calculate the spatial matrix of all audio input devices in the audio input array in each frequency dimension based on the active matrix A;

[0026] in, Represents the spatial matrix of all audio input devices in the audio input array in the fth frequency dimension, represents the conjugate transposed matrix of the active matrix A;

[0027] Determine beam weights and beam target signals based on a spatial matrix of all audio input devices in the audio input array in all frequency dimensions;

[0028] in, represents the beam weight of the audio input device at frequency f for the mth device label in the audio input array, represents the target signal steering vector for all audio input devices in the audio input array, represents the inverse of the conjugate transposed matrix of the active matrix A, Represents the conjugate transposed vector of the target signal steering vector.

[0029] According to an active matrix-based audio processing method provided by the present invention, the active matrix A is decomposed and mask optimization is performed to determine the mask target signal and the mask non-target signal, including:

[0030] Performing eigenvalue decomposition on the spatial matrix of each frequency dimension of all audio input devices in the audio input array to determine the eigenvector matrix and the diagonal eigenvalue matrix of the spatial matrix of each frequency dimension;

[0031] in, , N4 represents the space matrix The number of eigenvalues ​​of Representation space matrix The eigenvector matrix of Representation space matrix The diagonal eigenvalue matrix of Representation space matrix The conjugate transpose vector of the eigenvector matrix;

[0032] Determine the masked target signal and the masked non-target signal based on the eigenvector matrix, the diagonal eigenvalue matrix and the active matrix A of the space matrix of all frequency dimensions;

[0033] in, represents the mask non-target signal matrix, It represents the main target signal component after decomposing the audio input device of the mth device label at all t time points and all frequency dimensions. Representation space matrix The eigenvector of the largest eigenvalue in the eigenvector matrix, Display space matrix The eigenvectors of the matrix are the conjugate transpose of the eigenvector of the largest eigenvalue.

[0034] According to an active matrix-based audio processing method provided by the present invention, a first target signal is enhanced to determine a second target signal, and a first non-target signal is suppressed to determine a second non-target signal, including:

[0035] Performing time-frequency analysis on the first target signal, determining an enhancement strategy based on the analysis result, and performing signal enhancement on the first target signal based on the enhancement strategy to determine a second target signal;

[0036] A time-frequency analysis is performed on the first non-target signal, a suppression strategy is determined based on the analysis result, and signal suppression is performed on the first non-target signal based on the suppression strategy to determine a second non-target signal.

[0037] According to an active matrix-based audio processing method provided by the present invention, a second audio signal is determined based on a second target signal and a second non-target signal, comprising:

[0038] determining whether the second non-target signal does not affect the output of the second target signal, and if so, determining the second audio signal based on the second target signal;

[0039] Otherwise, the second target signal is enhanced again, and at the same time, the second non-target signal is suppressed again, and the second audio signal is determined based on the second enhanced second target signal.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] By constructing an active matrix through all the collected first audio signals, analyzing the active matrix, determining the first target signal and the first non-target signal of all the first audio signals, determining the second target signal and the second non-target signal, and determining the second audio signal, the quality and signal-to-noise ratio of the audio signal can be significantly improved, the audio processing effect can be improved, the accuracy of target sound source separation and the anti-interference ability can be improved, and the dynamic combination of target signal enhancement and interference signal suppression can be realized, which is more suitable for high-quality audio processing applications in complex acoustic environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 It is a flowchart of an active matrix-based audio processing method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Embodiment 1:

[0046] The embodiment of the present invention provides an audio processing method based on an active matrix, such as Figure 1 As shown, including:

[0047] 101: Collecting first audio signals based on all audio input devices, and constructing an active matrix based on all collected first audio signals;

[0048] 102: Analyze the active matrix to determine first target signals and first non-target signals of all first audio signals;

[0049] 103: performing signal enhancement on the first target signal to determine a second target signal, and performing signal suppression on the first non-target signal to determine a second non-target signal;

[0050] 104: Determine a second audio signal based on the second target signal and the second non-target signal;

[0051] The method of collecting first audio signals based on all audio input devices and constructing an active matrix based on all collected first audio signals includes:

[0052] Determine the three-dimensional coordinates and device label of each audio input device, and determine the audio input array based on the three-dimensional coordinates and device labels of all audio input devices;

[0053] Collecting and preprocessing the first audio sub-signal of each audio input device, and determining the first audio signal based on the preprocessed first audio sub-signals of all audio input devices collected at the same time point;

[0054] Determine a sampling frequency, and determine a time step vector of all first audio sub-signals in the first audio signal based on the sampling frequency, wherein the time step vector includes a plurality of time points;

[0055] Determine a time domain vector of an audio input device corresponding to each first audio sub-signal based on a time domain signal of each first audio sub-signal in the first audio signal at each time point in the time step vector;

[0056] Determine a frame length and a window function of a short-time Fourier transform, perform a short-time Fourier transform on a time domain vector of an audio input device corresponding to each first audio sub-signal based on the frame length and the window function, and determine a complex spectrum vector of the audio input device corresponding to each first audio sub-signal;

[0057] Constructing an active matrix A based on the time step vector, the complex spectrum vectors of the audio input devices corresponding to all the first audio sub-signals, and the device labels of all the audio input devices;

[0058] in, 1, N1 represents the number of device tags, , N2 represents the number of time points in the time step vector, , N3 represents the number of frequency dimensions, Represents the complex spectrum value of the audio input device with the mth device tag at the tth time point and the fth frequency dimension, The complex spectrum vectors representing the audio input devices of the mth device label.

[0059] In this embodiment, the audio input device may be a microphone, a musical instrument interface and pickup, an acoustic sensor, a recorder, a player or a communication device.

[0060] In this embodiment, the first audio signal represents initial audio data collected by multiple audio input devices, and may include a mixture of a target sound source signal and an interference signal.

[0061] In this embodiment, the first target signal represents an audio signal related to a target sound source extracted from the first audio signal.

[0062] In this embodiment, the first non-target signal represents an interference signal or background noise separated from the first audio signal.

[0063] In this embodiment, signal enhancement is a technical process for improving the quality and clarity of the target signal.

[0064] In this embodiment, signal suppression is a technical process of reducing the interference of non-target signals on target signals.

[0065] In this embodiment, the second target signal represents the target sound source signal after signal enhancement, and has higher quality.

[0066] In this embodiment, the second non-target signal represents an interference signal after signal suppression, and has lower interference.

[0067] In this embodiment, the physical space three-dimensional coordinates of each audio input device are determined to identify the precise position of the device, the audio input devices are numbered by device tags (device numbers) to determine the device tags, and an ordered audio input array is formed in combination with the three-dimensional coordinates to provide a position reference for subsequent spatial signal processing.

[0068] In this embodiment, each audio input device collects the original first audio sub-signal and pre-processes it (such as denoising and filtering). Based on multiple audio sub-signals collected at the same time point, a complete first audio signal is synthesized to ensure multi-device synchronization and signal consistency.

[0069] In this embodiment, a sampling frequency (the number of signal samples collected per second) is determined for digitizing the analog signal.

[0070] In this embodiment, the audio input array may be a linear array, a circular array, a rectangular grid, or other custom shapes.

[0071] In this embodiment, based on the sampling frequency, a time step vector, ie, a sequence of sampling moments of each audio sub-signal at different time points, is calculated to provide time resolution for time domain signal analysis.

[0072] In this embodiment, for each sub-signal in the first audio signal, its time domain signal at each time point in the time step vector is extracted, and these time domain signals are combined with the three-dimensional position of the corresponding audio input device to generate a time domain vector for subsequent frequency domain analysis.

[0073] In this embodiment, the frame length (signal length of each conversion) and window function (such as a Hamming window, used to reduce boundary effects) of the short-time Fourier transform (STFT) are determined, the STFT is applied to the time domain vector, its performance in the frequency domain is obtained, and the corresponding complex spectrum vector is generated for analysis of frequency components and amplitudes.

[0074] In this embodiment, the data of multiple devices and multiple time points are organized into an active matrix using a time step vector, all complex spectrum vectors and device labels. The matrix represents the distribution of the sound source signal in the time, frequency and space dimensions, providing data support for multi-dimensional signal separation and processing.

[0075] In this embodiment, the active matrix A is a three-dimensional matrix that integrates audio signal information in time, frequency, and space dimensions.

[0076] In this embodiment, each audio input device corresponds to a complex frequency spectrum vector, which represents the performance of the signal in the frequency domain and is used to analyze the frequency characteristics of the signal.

[0077] The beneficial effects of the above technical solution are as follows: by constructing an active matrix through all the collected first audio signals, analyzing the active matrix, determining the first target signal and the first non-target signal of all the first audio signals, determining the second target signal and the second non-target signal, and determining the second audio signal, the quality and signal-to-noise ratio of the audio signal can be significantly improved, the audio processing effect can be improved, the accuracy of target sound source separation and the anti-interference ability can be improved, and a dynamic combination of target signal enhancement and interference signal suppression can be achieved, which is more suitable for high-quality audio processing applications in complex acoustic environments.

[0078] Embodiment 2:

[0079] An embodiment of the present invention provides an active matrix-based audio processing method, which analyzes the active matrix and determines first target signals and first non-target signals of all first audio signals, including:

[0080] Directional filtering is performed on the active matrix A based on the audio input array to determine the beam target signal matrix;

[0081] Decomposing the active matrix A and optimizing the mask to determine the mask target signal matrix and the mask non-target signal matrix;

[0082] Determine a comprehensive target signal matrix based on the beam target signal matrix and the mask target signal matrix;

[0083] in, represents the comprehensive target signal matrix, represents the beam target signal matrix, represents the mask target signal matrix, Indicates that the fth frequency dimension belongs to the low-frequency dimension, Indicates that the fth frequency dimension belongs to the high-frequency dimension;

[0084] Performing an inverse short-time Fourier transform on the comprehensive target signal matrix to determine first target signals of all first audio signals;

[0085] An inverse short-time Fourier transform is performed on the masked non-target signal to determine first non-target signals of all first audio signals.

[0086] In this embodiment, the spatial position information of the audio input array is used to perform directional filtering on the active matrix A. The directional filtering focuses on the sound in a specific direction through an algorithm (such as beamforming) and suppresses interference in other directions. After directional filtering, the beam target signal matrix is ​​determined.

[0087] In this embodiment, the beam weight is a complex value.

[0088] In this embodiment, The target signal steering vector includes the relative amplitude and phase of each audio input device when the audio input array receives the target signal at frequency f, and the vector length is N1.

[0089] In this embodiment, the active matrix A is decomposed to separate the signal into different components, and the mask optimization technology is applied to further extract the mask target signal matrix and the mask non-target signal matrix.

[0090] In this embodiment, the beam target signal matrix and the mask target signal matrix are integrated to form a final integrated target signal matrix according to the high frequency or low frequency in the frequency dimension, combining the advantages of directional filtering and mask optimization to enhance the quality and integrity of the target signal.

[0091] In this embodiment, an inverse short-time Fourier transform (iSTFT) is performed on the integrated target signal matrix to restore the frequency domain signal to the time domain signal, thereby generating the first target signals (optimized target signals) of all the first audio signals.

[0092] In this embodiment, iSTFT is performed on the mask non-target signal matrix to generate first non-target signals (optimized non-target signals) of all first audio signals.

[0093] The beneficial effects of the above technical solution are: decomposing the active matrix to determine the first target signal and the first non-target signal of all the first audio signals can improve the accuracy of target signal separation and anti-interference ability, is suitable for complex multi-sound source scenarios, balances directivity and frequency characteristics, and significantly improves audio processing effects and signal quality.

[0094] Embodiment 3:

[0095] The embodiment of the present invention provides an audio processing method based on an active matrix, which performs directional filtering on an active matrix A based on an audio input array to determine a beam target signal, including:

[0096] Calculate the spatial matrix of all audio input devices in the audio input array in each frequency dimension based on the active matrix A;

[0097] in, Represents the spatial matrix of all audio input devices in the audio input array in the fth frequency dimension, represents the conjugate transposed matrix of the active matrix A;

[0098] Determine beam weights and beam target signals based on a spatial matrix of all audio input devices in the audio input array in all frequency dimensions;

[0099] in, represents the beam weight of the audio input device at frequency f for the mth device label in the audio input array, represents the target signal steering vector for all audio input devices in the audio input array, represents the inverse of the conjugate transposed matrix of the active matrix A, Represents the conjugate transposed vector of the target signal steering vector.

[0100] In this embodiment, for each frequency dimension, a matrix reflecting the spatial relationship between the device and the sound source is generated, which is called a spatial matrix.

[0101] In this embodiment, the beam weight is used to enhance the signal in the target direction and suppress interference in other directions. The calculated beam weight is adapted to all frequencies, thereby achieving directivity enhancement in the entire frequency band.

[0102] In this embodiment, the active matrix A is weighted using the calculated beam weights to extract the signal reaching the audio input array from the target direction to generate a beam target signal, which only retains the sound in the target direction and effectively suppresses interference from other directions, thereby ensuring the clarity and accuracy of the target sound source.

[0103] The beneficial effects of the above technical solution are as follows: based on the audio input array, the active matrix A is directional filtered to determine the beam target signal, which can achieve efficient extraction and enhancement of the target direction signal, adapt to complex multi-band scenes, improve the accuracy of target sound source separation and anti-interference ability, and provide more powerful technical support for applications such as speech recognition and far-field sound pickup.

[0104] Embodiment 4:

[0105] The embodiment of the present invention provides an active matrix-based audio processing method, which decomposes the active matrix A and performs mask optimization to determine a mask target signal and a mask non-target signal, including:

[0106] Performing eigenvalue decomposition on the spatial matrix of each frequency dimension of all audio input devices in the audio input array to determine an eigenvector matrix and a diagonal eigenvalue matrix of the spatial matrix of each frequency dimension;

[0107] in, , N4 represents the space matrix The number of eigenvalues ​​of Representation space matrix The eigenvector matrix of Representation space matrix The diagonal eigenvalue matrix of Representation space matrix The conjugate transpose vector of the eigenvector matrix;

[0108] Determine the masked target signal and the masked non-target signal based on the eigenvector matrix, the diagonal eigenvalue matrix and the active matrix A of the space matrix of all frequency dimensions;

[0109] in, represents the mask non-target signal matrix, It represents the main target signal component after decomposing the audio input device of the mth device label at all t time points and all frequency dimensions. Representation space matrix The eigenvector of the largest eigenvalue in the eigenvector matrix, Display space matrix The eigenvectors of the matrix are the conjugate transpose of the eigenvector of the largest eigenvalue.

[0110] In this embodiment, Represents the non-main target signal components after decomposition of the audio input device of the mth device label at all t time points and all frequency dimensions.

[0111] In this embodiment, Represents the main signal components of the audio input devices of all device labels after decomposition at all t time points and all frequency dimensions.

[0112] In this embodiment, Represents the non-main target signal components after decomposition of the audio input devices of all device labels at all t time points and all frequency dimensions.

[0113] In this embodiment, each eigenvalue corresponds to an eigenvector in the eigenvector matrix. If each eigenvector in the eigenvector matrix includes M1 eigenvalues, the eigenvector matrix is ​​an M1×N4 matrix.

[0114] In this embodiment, the masked target signal is a signal in the target direction extracted by feature decomposition, with interference removed.

[0115] In this embodiment, the masked non-target signal: the interference direction signal extracted by feature decomposition is used for noise reduction or signal separation.

[0116] The beneficial effects of the above technical solution are as follows: decomposing the active matrix and optimizing the mask to determine the mask target signal and the mask non-target signal can improve the accuracy of target signal extraction, avoid multipath interference, and is suitable for speech enhancement and signal separation in high noise environments.

[0117] Embodiment 5:

[0118] An embodiment of the present invention provides an active matrix-based audio processing method, which enhances a first target signal to determine a second target signal, and suppresses a first non-target signal to determine a second non-target signal, including:

[0119] Performing time-frequency analysis on the first target signal, determining an enhancement strategy based on the analysis result, and performing signal enhancement on the first target signal based on the enhancement strategy to determine a second target signal;

[0120] A time-frequency analysis is performed on the first non-target signal, a suppression strategy is determined based on the analysis result, and signal suppression is performed on the first non-target signal based on the suppression strategy to determine a second non-target signal.

[0121] In this embodiment, the frequency distribution of the first target signal at different times is analyzed, and characteristics such as energy concentration areas and main frequency components are extracted to reveal the acoustic laws of the target signal.

[0122] In this embodiment, the noise characteristics, frequency distribution and interference direction pattern of the first non-target signal are analyzed to extract useful information for suppressing target noise.

[0123] In this embodiment, the enhancement strategy may be gain adjustment: amplifying a specific frequency band of the target signal; speech quality enhancement: such as applying Wiener filtering or spectral subtraction to further eliminate residual noise; nonlinear processing: such as dynamic range compression to optimize the dynamic characteristics of the signal.

[0124] In this embodiment, the suppression strategy can be noise mask processing: based on the time-frequency mask, the high noise area is significantly attenuated; spatial domain suppression: the spatial filtering characteristics of the array are used to suppress interference signals in a specific direction; low-pass filtering or band-stop filtering: the main frequency components of non-target signals are filtered out.

[0125] In this embodiment, an enhancement strategy is implemented on the first target signal to generate a second target signal, retaining the main components of the target sound source while eliminating interference and distortion; thereby improving speech clarity, naturalness and signal-to-noise ratio.

[0126] In this embodiment, a suppression strategy is implemented on the first non-target signal to generate a second non-target signal, thereby maximally weakening the noise component and reducing the masking effect on the target signal; and maintaining a smooth transition of the background environment sound.

[0127] The beneficial effects of the above technical solution are as follows: the first target signal is enhanced to determine the second target signal, and the first non-target signal is suppressed to determine the second non-target signal, which can improve the clarity of the target signal and significantly suppress noise interference.

[0128] Embodiment 6:

[0129] An embodiment of the present invention provides an active matrix-based audio processing method, which determines a second audio signal based on a second target signal and a second non-target signal, including:

[0130] determining whether the second non-target signal does not affect the output of the second target signal, and if so, determining the second audio signal based on the second target signal;

[0131] Otherwise, the second target signal is enhanced again, and at the same time, the second non-target signal is suppressed again, and the second audio signal is determined based on the second enhanced second target signal.

[0132] In this embodiment, the masking effect, signal-to-noise ratio and auditory quality of the second non-target signal on the second target signal are analyzed, and signal evaluation indicators such as target signal-to-interference ratio (SIR), speech clarity quality (PESQ), or correlation analysis of the target signal by an adaptive algorithm can be used.

[0133] In this embodiment, if it has no impact: it means that the second target signal has reached the requirement of being clear and usable, and directly proceeds to the next step of processing.

[0134] In this embodiment, if the impact is: it means that the interference signal still masks the target signal, and it is necessary to further optimize the enhancement and suppression strategies, analyze the impact of the remaining interference on the target signal, and further enhance the main frequency range or speech characteristics of the target signal; extract the remaining characteristics of the interference signal and dynamically adjust the suppression strategy.

[0135] In this embodiment, after determining that the interference of the second non-target signal on the second target signal has dropped to an acceptable level, the second enhanced target signal is used as the main component to determine the final second audio signal:

[0136] The beneficial effects of the above technical solution are as follows: the second audio signal is determined based on the second target signal and the second non-target signal, and a dynamic feedback optimization mechanism is constructed to ensure that the output audio signal always has the highest voice quality and the lowest interference in a complex environment.

[0137] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0138] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An audio processing method based on active matrix, characterized in that: include: 101: Collecting first audio signals based on all audio input devices, and constructing an active matrix based on all collected first audio signals; 102: Analyze the active matrix to determine first target signals and first non-target signals of all first audio signals; 103: performing signal enhancement on the first target signal to determine a second target signal, and performing signal suppression on the first non-target signal to determine a second non-target signal; 104: Determine a second audio signal based on the second target signal and the second non-target signal; The method of collecting first audio signals based on all audio input devices and constructing an active matrix based on all collected first audio signals includes: Determine the three-dimensional coordinates and device label of each audio input device, and determine the audio input array based on the three-dimensional coordinates and device labels of all audio input devices; Collecting and preprocessing the first audio sub-signal of each audio input device, and determining the first audio signal based on the preprocessed first audio sub-signals of all audio input devices collected at the same time point; Determine a sampling frequency, and determine a time step vector of all first audio sub-signals in the first audio signal based on the sampling frequency, wherein the time step vector includes a plurality of time points; Determine a time domain vector of an audio input device corresponding to each first audio sub-signal based on a time domain signal of each first audio sub-signal in the first audio signal at each time point in the time step vector; Determine a frame length and a window function of a short-time Fourier transform, perform a short-time Fourier transform on a time domain vector of an audio input device corresponding to each first audio sub-signal based on the frame length and the window function, and determine a complex spectrum vector of the audio input device corresponding to each first audio sub-signal; Constructing an active matrix A based on the time step vector, the complex spectrum vectors of the audio input devices corresponding to all the first audio sub-signals, and the device labels of all the audio input devices; in, , N1 represents the number of device tags, , N2 represents the number of time points in the time step vector, , N3 represents the number of frequency dimensions, Represents the complex spectrum value of the audio input device with the mth device tag at the tth time point and the fth frequency dimension, The complex spectrum vectors representing the audio input devices of the mth device label respectively; The step of analyzing the active matrix to determine the first target signal and the first non-target signal of all the first audio signals includes: Directional filtering is performed on the active matrix A based on the audio input array to determine the beam target signal matrix; Decomposing the active matrix A and optimizing the mask to determine the mask target signal matrix and the mask non-target signal matrix; Determine a comprehensive target signal matrix based on the beam target signal matrix and the mask target signal matrix; in, represents the comprehensive target signal matrix, represents the beam target signal matrix, represents the mask target signal matrix, Indicates that the fth frequency dimension belongs to the low-frequency dimension, Indicates that the fth frequency dimension belongs to the high-frequency dimension; Performing an inverse short-time Fourier transform on the comprehensive target signal matrix to determine first target signals of all first audio signals; An inverse short-time Fourier transform is performed on the masked non-target signal to determine first non-target signals of all first audio signals.

2. The active matrix-based audio processing method according to claim 1, characterized in that: Directional filtering of the active matrix A is performed based on the audio input array to determine the beam target signal, including: Calculate the spatial matrix of all audio input devices in the audio input array in each frequency dimension based on the active matrix A; in, Represents the spatial matrix of all audio input devices in the audio input array in the fth frequency dimension, represents the conjugate transposed matrix of the active matrix A; Determine beam weights and beam target signals based on a spatial matrix of all audio input devices in the audio input array in all frequency dimensions; in, represents the beam weight of the audio input device at frequency f for the mth device label in the audio input array, represents the target signal steering vector for all audio input devices in the audio input array, represents the inverse of the conjugate transposed matrix of the active matrix A, Represents the conjugate transposed vector of the target signal steering vector.

3. The active matrix-based audio processing method according to claim 2, characterized in that: The active matrix A is decomposed and the mask is optimized to determine the mask target signal and the mask non-target signal, including: Performing eigenvalue decomposition on the spatial matrix of each frequency dimension of all audio input devices in the audio input array to determine the eigenvector matrix and the diagonal eigenvalue matrix of the spatial matrix of each frequency dimension; in, , N4 represents the space matrix The number of eigenvalues ​​of Representation space matrix The eigenvector matrix of Representation space matrix The diagonal eigenvalue matrix of Representation space matrix The conjugate transpose vector of the eigenvector matrix; Determine the masked target signal and the masked non-target signal based on the eigenvector matrix, the diagonal eigenvalue matrix and the active matrix A of the space matrix of all frequency dimensions; in, represents the mask non-target signal matrix, It represents the main target signal component after decomposing the audio input device of the mth device label at all t time points and all frequency dimensions. Representation space matrix The eigenvector of the largest eigenvalue in the eigenvector matrix, Display space matrix The eigenvectors of the matrix are the conjugate transpose of the eigenvector of the largest eigenvalue.

4. The active matrix-based audio processing method according to claim 1, characterized in that: The method of enhancing the first target signal to determine the second target signal and suppressing the first non-target signal to determine the second non-target signal includes: Performing time-frequency analysis on the first target signal, determining an enhancement strategy based on the analysis result, and performing signal enhancement on the first target signal based on the enhancement strategy to determine a second target signal; A time-frequency analysis is performed on the first non-target signal, a suppression strategy is determined based on the analysis result, and signal suppression is performed on the first non-target signal based on the suppression strategy to determine a second non-target signal.

5. The active matrix-based audio processing method according to claim 1, characterized in that: Determining a second audio signal based on the second target signal and the second non-target signal includes: determining whether the second non-target signal does not affect the output of the second target signal, and if so, determining the second audio signal based on the second target signal; Otherwise, the second target signal is enhanced again, and at the same time, the second non-target signal is suppressed again, and the second audio signal is determined based on the second enhanced second target signal.

Images