Multi-channel audio data enhancement processing methods, devices, equipment and media

Abstract

This disclosure provides a method, apparatus, device, and medium for multi-channel audio data enhancement processing, relating to the field of audio processing technology. The method includes: acquiring audio data to be enhanced from multiple channels; performing a short-time Fourier transform on the audio data of each channel to obtain time-spectrum data corresponding to each channel; generating a feature matrix of the audio data based on the time-spectrum data of each channel; clustering the elements in the feature matrix according to the number of sound sources to obtain multiple sound source sets containing one or more elements; sorting the elements in each sound source set according to the ascending order of time-frequency signal energy, removing a predetermined proportion of the elements at the top of each sound source set to obtain a feature matrix after interference removal; and performing a short-time inverse Fourier transform on the interference-removed feature matrix to obtain the enhanced audio data. This disclosure not only improves the enhancement effect of audio data but also has a faster processing speed.

Description

Technical Field

[0001] This disclosure relates to the field of audio processing technology, and in particular to a multi-channel audio data enhancement processing method, apparatus, device, and medium. Background Technology

[0002] Since audio data obtained in real life is inevitably affected by environmental noise, which often contains various interference components, the effective components in the audio data are covered, making it difficult for people to obtain the semantic information contained in the audio data. To address this, people usually use audio processing technology to obtain the effective parts of the audio data as accurately as possible.

[0003] Audio processing is further subdivided into multi-channel audio processing and single-channel audio processing based on the number of channels. Multi-channel audio processing refers to processing audio data acquired by multiple voice acquisition devices. It typically employs beamforming technology, a technique for directional speech enhancement, which acts as a spatial filter to extract the target signal from the mixed signal received by the audio acquisition devices.

[0004] However, the relevant technology identifies the part with the highest energy as the direction of target audio processing, which is insufficient in suppressing interference and noise. As a result, the technology performs poorly in audio processing in noisy environments with strong interference.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] This disclosure provides a multi-channel audio data enhancement processing method, apparatus, device, and medium, which at least to some extent overcomes the technical problem that multi-channel audio data enhancement schemes provided in related technologies have poor performance in noisy environments.

[0007] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0008] According to one aspect of this disclosure, a multi-channel audio data enhancement processing method is provided, comprising: acquiring audio data to be enhanced, wherein the audio data includes audio data acquired from multiple channels; performing a short-time Fourier transform on the audio data of each channel to obtain time-frequency data corresponding to each channel; generating a feature matrix of the audio data based on the time-frequency data of each channel, wherein each element in the feature matrix corresponds to a time-frequency point; clustering the elements in the feature matrix according to the number of sound sources to obtain multiple sound source sets, wherein each sound source set contains one or more elements; sorting the elements in each sound source set according to the time-frequency point signal energy from smallest to largest, removing a predetermined proportion of the elements at the top of each sound source set to obtain a feature matrix after removing interference; and performing a short-time inverse Fourier transform on the feature matrix after removing interference to obtain enhanced audio data.

[0009] In some embodiments, before performing a short-time Fourier transform on the audio data of each channel to obtain the time-spectrum data corresponding to each channel, the method further includes:

[0010] The audio data to be enhanced is normalized using the following formula:

[0011]

[0012] Where X' represents the normalized audio data, X represents the audio data before normalization, E(X) represents the expectation of the audio data X, and Var(X) represents the variance of the audio data X.

[0013] In some embodiments, clustering the elements in the feature matrix according to the number of sound sources includes: using the K-means algorithm to cluster the elements in the feature matrix according to the number of sound sources.

[0014] In some embodiments, before clustering the elements in the feature matrix according to the number of sound sources to obtain multiple sound source sets, the method further includes: subtracting a preset background noise value from each element in the feature matrix to obtain a feature matrix after removing background noise; wherein, clustering the elements in the feature matrix according to the number of sound sources to obtain multiple sound source sets includes: clustering the elements in the feature matrix after removing background noise according to the number of sound sources to obtain multiple sound source sets.

[0015] In some embodiments, the time-spectrum data of each channel includes amplitude spectrum data and phase spectrum data; wherein, generating the feature matrix of the audio data based on the time-spectrum data of each channel includes: calculating the amplitude spectrum cross-correlation matrix between different channels based on the amplitude spectrum data; calculating the phase spectrum cross-correlation matrix between different channels based on the phase spectrum data; and generating the feature matrix based on the amplitude spectrum cross-correlation matrix and the phase spectrum cross-correlation matrix.

[0016] In some embodiments, the amplitude spectrum cross-correlation matrix between different channels is calculated using the following formula:

[0017] r i,j =X a,i -X a,j

[0018] The phase spectrum cross-correlation matrix between different channels is calculated using the following formula:

[0019] s i,j =X p,i -X p,j

[0020] The characteristic matrix is ​​calculated using the following formula:

[0021]

[0022] Where i and j represent channel numbers, which are natural numbers; a represents amplitude; p represents phase; X a,i X represents the amplitude spectrum matrix of the i-th channel; a,j Let r be the amplitude spectrum matrix of the j-th channel; i,j X represents the amplitude spectrum cross-correlation matrix between the i-th and j-th channels; p,i X is the phase spectrum matrix of the i-th channel; p,j Let be the phase spectrum matrix of the j-th channel; s i,j Let f be the cross-correlation matrix between the phase spectra of the i-th and j-th channels; f represents the characteristic matrix. It represents the Hadamardi (or Hadama) stack.

[0023] In some embodiments, performing a short-time inverse Fourier transform on the interference-free feature matrix to obtain enhanced audio data includes: generating interference-free time-spectrum data based on the interference-free feature matrix; and performing a short-time inverse Fourier transform on the interference-free time-spectrum data to obtain enhanced audio data.

[0024] In some embodiments, before performing a short-time inverse Fourier transform on the interference-removed feature matrix to obtain enhanced audio data, the method further includes: transforming the interference-removed feature matrix using a softmax function to obtain a normalized feature matrix; wherein, performing a short-time inverse Fourier transform on the interference-removed feature matrix to obtain enhanced audio data includes: performing a short-time inverse Fourier transform on the normalized feature matrix to obtain enhanced audio data.

[0025] According to another aspect of this disclosure, an apparatus is also provided, comprising: an audio data acquisition module for acquiring audio data to be enhanced, wherein the audio data includes audio data acquired from multiple channels; a time-spectrum data generation module for performing a short-time Fourier transform on the audio data of each channel to obtain time-spectrum data corresponding to each channel; a feature matrix generation module for generating a feature matrix of the audio data based on the time-spectrum data of each channel, wherein each element in the feature matrix corresponds to a time-frequency point; a clustering module for clustering the elements in the feature matrix according to the number of sound sources to obtain multiple sound source sets, wherein each sound source set contains one or more elements; an interference removal module for sorting the elements in each sound source set in ascending order of time-frequency point signal energy, removing a predetermined proportion of the elements at the top of each sound source set, to obtain an interference-removed feature matrix; and an audio enhancement result output module for performing a short-time inverse Fourier transform on the interference-removed feature matrix to obtain enhanced audio data.

[0026] According to another aspect of this disclosure, an electronic device is also provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the multi-channel audio data enhancement processing method described in any one of the preceding claims by executing the executable instructions.

[0027] According to another aspect of this disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the multi-channel audio data enhancement processing method described in any one of the preceding claims.

[0028] According to another aspect of this disclosure, a computer program product is also provided, including a computer program that, when executed by a processor, implements the multi-channel audio data enhancement processing method of any one of the above.

[0029] The multi-channel audio data enhancement processing method, apparatus, device, and medium provided in this disclosure, after acquiring the multi-channel audio data to be enhanced, performs a short-time Fourier transform on the audio data of each channel to obtain the time-spectrum data corresponding to each channel. Then, based on the time-spectrum data of each channel, a feature matrix of the audio data to be enhanced is generated. The elements in the feature matrix are clustered according to the number of sound sources, so that the elements belonging to each sound source are aggregated into a sound source set. The elements in each sound source set are sorted according to the time-frequency energy of each element. The elements at the top of the sorted sequence in each sound source set are removed to obtain the feature matrix after interference removal. Finally, a short-time inverse Fourier transform is performed on the feature matrix after interference removal to obtain the enhanced audio data.

[0030] This embodiment employs a clustering algorithm to cluster data from various audio sources and removes similar interference components from each source, achieving good audio enhancement in noisy environments. Furthermore, using a feature matrix composed of inter-channel cross-correlation matrices as input to the clustering method improves audio data processing speed while consuming fewer computational resources, thus possessing high practical value.

[0031] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0032] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0033] Figure 1 A schematic diagram of an exemplary application system architecture for a multi-channel audio data enhancement processing method according to an embodiment of this disclosure is shown.

[0034] Figure 2 A schematic diagram of audio data carrying interference components is shown in an embodiment of this disclosure;

[0035] Figure 3 This illustration shows a schematic diagram of the acquisition principle of multi-channel audio data in an embodiment of the present disclosure;

[0036] Figure 4 This diagram illustrates a flowchart of a multi-channel audio data enhancement processing method according to an embodiment of the present disclosure.

[0037] Figure 5 This diagram illustrates a flowchart of a feature matrix generation method according to an embodiment of the present disclosure.

[0038] Figure 6 This diagram illustrates a flowchart of a multi-channel audio data enhancement processing method according to an embodiment of the present disclosure.

[0039] Figure 7 This diagram illustrates a detailed implementation flowchart of a multi-channel audio data enhancement processing method according to an embodiment of the present disclosure.

[0040] Figure 8 This diagram illustrates a multi-channel audio data enhancement processing apparatus according to an embodiment of the present disclosure.

[0041] Figure 9 A block diagram of an electronic device according to an embodiment of the present disclosure is shown; and

[0042] Figure 10 A schematic diagram of a computer-readable storage medium according to an embodiment of the present disclosure is shown. Detailed Implementation

[0043] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0044] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0045] The specific implementation methods of the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0046] Figure 1 This illustration shows an exemplary application system architecture diagram of a multi-channel audio data enhancement processing method according to an embodiment of the present disclosure, such as... Figure 1 As shown, the system architecture may include an audio acquisition device 101 and an audio processing device 102.

[0047] It should be noted that the audio acquisition device 101 mentioned above can be various audio acquisition devices, including but not limited to microphones, recorders, voice recorders, smartphones, and other terminal devices with recording or audio acquisition functions. The audio processing device 102 mentioned above can be a terminal device or server with audio data processing functions.

[0048] Optionally, the terminal device can be various electronic devices, including but not limited to smartphones, desktop computers, laptops, etc.

[0049] Optionally, the server can be a server that provides various services, such as a backend management server that supports the device operated by the user using the audio acquisition device 101. The backend management server can analyze and process received requests and other data, and feed the processing results back to the terminal device. In specific implementations, the server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The terminal and the server can be connected directly or indirectly through wired or wireless communication, and this disclosure does not impose any restrictions.

[0050] It should be noted that the audio acquisition device 101 and the audio processing device 102 described above can be the same device or different devices. When the audio acquisition device 101 and the audio processing device 102 are different devices, the audio acquisition device 101 and the audio processing device 102 can be connected via a wired network or a wireless network.

[0051] Those skilled in the art will know that Figure 1 The number of audio acquisition devices and audio processing devices shown is merely illustrative; any number of audio acquisition devices and audio processing devices can be used depending on actual needs. This disclosure does not limit this.

[0052] This disclosure provides a multi-channel audio data enhancement processing method, which can be executed by any electronic device with computing power.

[0053] In some embodiments, the multi-channel audio data enhancement processing method provided in this disclosure can be derived from... Figure 1 The audio processing device shown is used to perform this; in other embodiments, it may be performed by... Figure 1 The audio processing device and the audio acquisition device shown are implemented through interaction.

[0054] Figure 2 This illustration shows a schematic diagram of audio data carrying interference components according to an embodiment of the present disclosure, such as... Figure 2 As shown, after being interfered with by noise in the environment, the pure audio data containing only semantic information is covered by the interference components, making the audio data finally acquired by the audio acquisition device greatly affected by noise, and making it difficult to easily identify the semantic information in the audio data.

[0055] Figure 3 This illustration shows a schematic diagram of the acquisition principle of multi-channel audio data in an embodiment of this disclosure, such as... Figure 3 As shown, each circle represents a microphone, and the angle of incidence of the sound is θ. i (Angle between the incident direction and the microphone array normal), where d is the distance between the two microphones. The received signal can be represented as:

[0056] X(t) = AS(t) + N(t)

[0057] X(t) is the received signal, S(t) is the target audio signal, N(t) is the interference component, and A is the steering vector. It should be noted that... Figure 3 The types and quantities of audio acquisition devices described are merely illustrative. Any type and number of audio acquisition devices can be used according to actual needs, and this disclosure does not impose any specific limitations on them.

[0058] Figure 4 This diagram illustrates a flowchart of a multi-channel audio data enhancement processing method according to an embodiment of the present disclosure, such as... Figure 4 As shown, the method includes the following steps:

[0059] S402, acquire the audio data to be enhanced, which includes audio data acquired from multiple channels.

[0060] It should be noted that the audio data to be enhanced can be any type of audio data to be enhanced, including but not limited to voice data. In some embodiments, it can be multi-channel audio data acquired by multiple audio acquisition devices (e.g., microphones, recorders, etc.), or multi-channel audio data imported from other devices. The method of acquiring the audio data to be enhanced is not specifically limited in the embodiments of this disclosure.

[0061] S404 performs a short-time Fourier transform on the audio data of each channel to obtain the time-spectrum data corresponding to each channel.

[0062] It should be noted that the short-time Fourier transform is a transformation that converts a time-domain signal to the time-frequency domain. After performing a short-time Fourier transform on the audio data of each channel, the time-spectrum data corresponding to each channel can be obtained. The time-spectrum data contains two dimensions: the time domain and the frequency domain.

[0063] In some embodiments, the audio data of each channel can be subjected to a short-time Fourier transform using the following formula:

[0064]

[0065] Where w(t-τ0) represents the window function centered at τ0, X(t) represents the time-domain audio signal, X(ω) represents the spectral-domain audio signal, j represents the complex unit, ω represents the angular frequency, and t represents time.

[0066] In one embodiment of this disclosure, prior to S404 above, the multi-channel audio data enhancement processing method provided in this embodiment can also be normalized using the following formula:

[0067]

[0068] Where X' represents the normalized audio data, X represents the unnormalized audio data, E(X) represents the expectation of the audio data, and Var(X) represents the variance of the audio data.

[0069] Through the above embodiments, normalizing the audio data ensures consistent data dimensions, prevents data shift, and facilitates subsequent short-time Fourier transform, resulting in more accurate time-spectrum data. It should be noted that the preprocessing method for the enhanced audio data in this disclosure is not specifically limited.

[0070] S406 generates a feature matrix of audio data based on the time-frequency data of each channel, where each element in the feature matrix corresponds to a time-frequency point.

[0071] It should be noted that the time-spectrum data for each channel includes amplitude spectrum data and phase spectrum data. The amplitude spectrum data contains the amplitude and frequency data of the channel's time-spectrum, while the phase spectrum data contains the phase and frequency data of the channel's time-spectrum. The characteristic matrix is ​​calculated from the cross-correlation matrix of the amplitude and phase spectrum data between channels.

[0072] S408, cluster the elements in the feature matrix according to the number of sound sources to obtain multiple sound source sets, wherein each sound source set contains one or more elements.

[0073] It should be noted that the elements in the feature matrix are clustered according to the number of sound sources, and similar elements are grouped together to form a sound source set. Each sound source set can consist of one or more elements.

[0074] In one embodiment of this disclosure, the K-means algorithm can be used to cluster the elements in the feature matrix based on the number of sound sources. Of course, other clustering algorithms can also be used to cluster the elements in the feature matrix, and this disclosure does not specifically limit the clustering algorithm.

[0075] In one embodiment of this disclosure, before clustering the elements in the feature matrix according to the number of sound sources to obtain multiple sound source sets, the method of this embodiment further includes: subtracting a preset background noise value from each element in the feature matrix to obtain a feature matrix after removing background noise. The method for removing background noise can first perform simple noise reduction processing on the feature matrix, and then perform clustering based on the noise-reduced feature matrix. This is beneficial in reducing the noise content of the data within the sound source sets, thus facilitating the final achievement of better audio processing results.

[0076] In one embodiment of this disclosure, after subtracting a preset background noise value from each element in the feature matrix to obtain a background noise-removed feature matrix, the elements in the background noise-removed feature matrix can be clustered according to the number of sound sources to obtain multiple sound source sets.

[0077] It should be noted that the preset background noise value in the embodiments of this disclosure refers to a pre-configured threshold that can represent the magnitude of background noise. Different thresholds can be configured according to different scenarios. In some embodiments, the noise energy value 1e-2 in the default time spectrum can be used. Of course, other values ​​can also be used as the preset background noise value. The embodiments of this disclosure do not specifically limit the preset background noise value.

[0078] S410 sorts the elements in each sound source set according to the order of time-frequency signal energy from small to large, removes the elements at the top of the sorted set by a preset proportion, and obtains the feature matrix after removing interference.

[0079] It should be noted that the preset ratio in the embodiments of this disclosure refers to a pre-configured threshold that can be used to filter interference points. It is configured in the form of a percentage. Different thresholds can be configured according to different scenarios. In some embodiments, the preset ratio used to remove interference is 40%. Of course, other values ​​can also be used as the preset ratio. The embodiments of this disclosure do not specifically limit the preset ratio for removing interference.

[0080] For example, the `reshape()` function can be used. This function transforms a specified matrix into a matrix of a specific dimension. The clustered feature matrix is ​​placed into the `reshape()` function to reshape it into a one-dimensional matrix. This one-dimensional matrix is ​​then fed into the histogram generation function `np.histogram()` to perform histogram statistics, arranging the elements in ascending order of energy. The energy of all elements is summed and multiplied by 40% to obtain a value, which is used as a threshold. An accumulator is set up, initially with a value of 0. Elements from the one-dimensional matrix are added to the accumulator in ascending order of energy until the accumulator value is equal to or just greater than the threshold. The calculation stops, and the elements in the one-dimensional matrix added to the accumulator are set to zero. Finally, the processed one-dimensional matrix is ​​restored to obtain the feature matrix after removing interference.

[0081] S412, perform a short-time inverse Fourier transform on the feature matrix after removing interference to obtain the enhanced audio data.

[0082] In one embodiment of this disclosure, the short-time inverse Fourier transform is calculated using the following formula:

[0083]

[0084] Where X(t) represents the time-domain audio signal, X(ω) represents the frequency-domain audio signal, j represents the complex unit, ω represents the angular frequency, and t represents time.

[0085] In one embodiment of this disclosure, the above-mentioned S412 can be implemented by the following steps: generating interference-free time-spectrum data based on the feature matrix after interference removal; performing short-time inverse Fourier transform on the interference-free time-spectrum data to obtain enhanced audio data.

[0086] In one embodiment of this disclosure, before performing a short-time inverse Fourier transform on the interference-removed feature matrix to obtain enhanced audio data, the method further includes: transforming the interference-removed feature matrix using a normalized exponential function (softmax) to obtain a normalized feature matrix; wherein, performing a short-time inverse Fourier transform on the interference-removed feature matrix to obtain enhanced audio data includes: performing a short-time inverse Fourier transform on the normalized feature matrix to obtain enhanced audio data. This embodiment of the disclosure normalizes the interference-removed feature matrix, which can widen the gap between large differences in values, ensuring that the total energy of the original audio data remains unchanged, thereby achieving better audio processing results.

[0087] In one embodiment of this disclosure, the softmax function is calculated using the following formula:

[0088]

[0089] Where i and k represent index values, which are natural numbers; K represents the number of elements in the feature matrix; x i x represents the i-th element in the characteristic matrix; k Represents the k-th element in the characteristic matrix; softmax(x) i ) represents the i-th element in the normalized feature matrix.

[0090] As described above, the multi-channel audio data processing method of this disclosure, after acquiring the multi-channel audio data to be enhanced, performs a short-time Fourier transform on the audio data of each channel to obtain the time-spectrum data corresponding to each channel. Then, based on the time-spectrum data of each channel, a feature matrix of the audio data to be enhanced is generated. The elements in the feature matrix are clustered according to the number of sound sources, so that elements belonging to each sound source are aggregated into a sound source set. The elements in each sound source set are sorted according to the time-frequency energy of each element. Elements at the top of the sorted sequence in each sound source set are removed by a predetermined proportion to obtain the interference-free feature matrix. Finally, a short-time inverse Fourier transform is performed on the interference-free feature matrix to obtain the enhanced audio data. By employing a clustering algorithm to obtain each sound source set and removing similar interference components from each sound source, better audio processing results can be achieved in noisy environments. In this disclosure embodiment, the feature matrix composed of the inter-channel cross-correlation matrix is ​​used as the input to the clustering method, ensuring the efficiency of the audio processing technology and thus solving the problem in the prior art where accurate audio processing cannot be performed due to environmental noise interference.

[0091] Figure 5 A flowchart of a feature matrix generation method according to an embodiment of this disclosure is shown, such as... Figure 5 As shown, the specific steps include the following:

[0092] S502 obtains the time-spectrum data for each channel.

[0093] In one embodiment of this disclosure, the time-spectrum data for each channel includes amplitude spectrum data and phase spectrum data.

[0094] The amplitude spectrum data is calculated using the following formula:

[0095] X a =|X(ω)| (5)

[0096] Phase spectrum data are calculated using the following formula:

[0097]

[0098] Where a represents amplitude; p represents phase; X a Indicates amplitude spectrum data; Xp X(ω) represents the phase spectrum data; X(ω) represents the audio signal in the frequency domain.

[0099] S504, calculates the amplitude spectrum cross-correlation matrix between different channels based on the amplitude spectrum data.

[0100] In one embodiment of this disclosure, the amplitude spectrum cross-correlation matrix between different channels is calculated using the following formula:

[0101] r i,j =X a,i -X a,j (7)

[0102] Where i and j represent channel numbers, and their values ​​are natural numbers; X a,i X represents the amplitude spectrum matrix of the i-th channel; a,j Represents the amplitude spectrum matrix of the j-th channel; e i,j Let represent the amplitude spectrum cross-correlation matrix between the i-th channel and the j-th channel.

[0103] S506, Calculate the phase spectrum cross-correlation matrix between different channels based on the phase spectrum data.

[0104] In one embodiment of this disclosure, the phase spectrum cross-correlation matrix between different channels is calculated using the following formula:

[0105] s i,j =X p,i -X p,j (8)

[0106] Among them, X p,i X represents the phase spectrum matrix of the i-th channel; p,j s represents the phase spectrum matrix of the j-th channel; i,j This represents the cross-correlation matrix between the phase spectra of the i-th and j-th channels.

[0107] It should be noted that the order of S504 and S506 can be interchanged or performed simultaneously, and this disclosure does not impose any specific restrictions on this.

[0108] S508 generates the characteristic matrix based on the amplitude spectrum cross-correlation matrix and the phase spectrum cross-correlation matrix.

[0109] In one embodiment of this disclosure, the feature matrix is ​​calculated using the following formula:

[0110]

[0111] Where f represents the characteristic matrix; It represents the Hadamardi (or Hadama) stack.

[0112] As described above, the feature matrix generation method of this embodiment calculates the amplitude spectrum data and phase spectrum data of each channel using the time-spectrum data of each channel. Based on the amplitude spectrum data and phase spectrum data of different channels, the amplitude spectrum cross-correlation matrix and phase spectrum cross-correlation matrix between different channels are calculated. Then, based on the amplitude spectrum cross-correlation matrix and phase spectrum cross-correlation matrix between corresponding channels, the feature matrix is ​​finally generated. Using the feature matrix as input for clustering eliminates the need for network modeling, resulting in faster audio processing speed while consuming fewer computational resources.

[0113] Figure 6 This diagram illustrates a flowchart of a multi-channel audio data enhancement processing method according to an embodiment of the present disclosure, such as... Figure 6 As shown, the specific steps include the following:

[0114] S602 preprocesses the elements in the feature matrix after removing background noise.

[0115] In one embodiment of this disclosure, the preprocessing method may include standardization and outlier filtering.

[0116] In one embodiment of this disclosure, the elements in the feature matrix after removing background noise are standardized using the following formula:

[0117]

[0118] Where Y' represents an element in the feature matrix after standardization and removal of background noise, Y represents an element in the feature matrix before standardization and removal of background noise, E(Y) represents the expectation of the elements in the feature matrix after removing background noise, and Var(Y) represents the variance of the elements in the feature matrix after removing background noise.

[0119] In one embodiment of this disclosure, the outlier filtering method first sets a threshold. When an element in the feature matrix after removing background noise is less than the threshold, the element is set to zero. The threshold is a default value taken when the algorithm's performance is at its highest; this disclosure does not impose specific limitations on its value.

[0120] Preprocessing the elements of the feature matrix after removing background noise can improve the efficiency of subsequent calculations and the accuracy of clustering results. This disclosure does not specify the particular method of preprocessing.

[0121] It should be noted that step S602 above is optional.

[0122] S604: Randomly select the center of the audio source category based on the number of audio sources in the audio data to be enhanced.

[0123] In one embodiment of this disclosure, it is known that there are k sound sources in the audio data to be enhanced, and k category centers are randomly selected, denoted as... …, in, This represents the k-th category center at the 0th iteration.

[0124] S606 assigns the elements in the feature matrix after removing background noise to the nearest category center among the randomly selected sound source category centers.

[0125] In one embodiment of this disclosure, before assigning the elements of the feature matrix after removing background noise to the nearest category center among the randomly selected sound source category centers, the method of this embodiment further includes:

[0126] Define a loss function:

[0127]

[0128] Where i represents the index, which is a natural number; c represents the sound source category; μ represents the center of the sound source category; M represents the number of elements in the feature matrix after removing background noise; x i c represents the i-th element in the feature matrix after removing background noise; i Indicates the i-th sound source category; J(c,μ) represents the center of the i-th sound source category; J(c,μ) represents the sum of the squared distances of all data in a sound source to the center of that category.

[0129] In one embodiment of this disclosure, elements in the feature matrix after background noise removal are assigned to the nearest center using the following formula:

[0130]

[0131] Where i and k represent the sequence number, and t represents the iteration step, which takes the value of a natural number; This represents the i-th sound source category during the t-th iteration. Let represent the center of the k-th sound source category during the t-th iteration. The above formula indicates that the i-th element is reassigned to the nearest sound source category center.

[0132] S608, recalculate the category center of the sound source based on the nearest sound source category center.

[0133] In one embodiment of this disclosure, the category center of the sound source is recalculated based on the nearest sound source category center using the following formula:

[0134]

[0135] Where i and k represent the sequence number, and the minimum value is in S606 above. Let represent the i-th sound source category in the t-th iteration; b represents the number of sound source categories; μ represents the center of a sound source category. This represents the class center of the k-th class in the (t+1)-th iteration.

[0136] In one embodiment of this disclosure, S606 and S608 are repeated until a convergence condition is met, at which point J converges, and the clustered sets of sound sources are obtained.

[0137] The convergence condition can be a maximum number of iteration steps. For example, in this disclosure, the maximum number of iteration steps is set to 300. Of course, the maximum number of iteration steps can be changed according to actual needs. This disclosure does not make a specific limit on the maximum number of iteration steps. The convergence condition can also be to pause after J(c,μ) is less than a threshold. In this embodiment, the threshold is not specifically limited.

[0138] It should be noted that the embodiments disclosed herein do not specifically limit the convergence conditions.

[0139] As described above, the multi-channel audio data processing method of this disclosure first preprocesses the elements of the feature matrix after removing background noise, then randomly selects sound source category centers according to the categories in the audio data, defines a loss function, and iterates on it until a convergence condition is met, causing the loss function to converge. The iteration process involves first assigning each element of the feature matrix after removing background noise to the nearest sound source category center, and then recalculating the center of each sound source category for each element. The use of the K-means clustering algorithm makes the convergence speed of the sound source set faster, the computational efficiency higher, and less affected by environmental noise, thus achieving better audio processing results in noisy environments.

[0140] Figure 7 This diagram illustrates a specific implementation flowchart of a multi-channel audio data enhancement processing method according to an embodiment of the present disclosure, such as... Figure 7 As shown, the specific steps include the following:

[0141] S701, acquire the audio data to be enhanced.

[0142] In one embodiment of this disclosure, audio data to be enhanced, including data acquired from multiple channels, is obtained.

[0143] S702, preprocess the acquired audio data to be enhanced to obtain preprocessed audio data.

[0144] In one embodiment of this disclosure, preprocessing can be performed using formula (2) in a normalized manner. This disclosure does not specifically limit the preprocessing method.

[0145] S703 performs a short-time Fourier transform on the preprocessed audio data to obtain the time-spectrum data corresponding to each channel.

[0146] S704, based on the time-spectrum data corresponding to each channel, obtains the amplitude spectrum cross-correlation matrix between different channels, including S7041 and S7042.

[0147] S7041 obtains the corresponding amplitude spectrum data based on the time spectrum data of different channels.

[0148] In one embodiment of this disclosure, the corresponding amplitude spectrum data can be obtained by formula (5) based on the time spectrum data of different channels.

[0149] S7042 generates amplitude spectrum cross-correlation matrices between different channels based on amplitude spectrum data from different channels.

[0150] In one embodiment of this disclosure, the amplitude spectrum cross-correlation matrix between different channels can be generated by formula (7) based on the amplitude spectrum data of different channels.

[0151] S705, based on the time-spectrum data corresponding to each channel, obtains the phase spectrum cross-correlation matrix between different channels, including S7051 and S7051.

[0152] S7051 obtains the corresponding phase spectrum data based on the time spectrum data of different channels.

[0153] In one embodiment of this disclosure, the corresponding phase spectrum data can be obtained by formula (7) based on the time spectrum data of different channels.

[0154] S7052, Based on the phase spectrum data of different channels, generate the phase spectrum cross-correlation matrix between different channels. In one embodiment of this disclosure, based on the phase spectrum data of different channels, the phase spectrum cross-correlation matrix between different channels can be generated by formula (8).

[0155] S706 generates a feature matrix of audio data based on the amplitude spectrum cross-correlation matrix and the phase spectrum cross-correlation matrix between different channels. Each element in the feature matrix corresponds to a time-frequency point.

[0156] S707, subtract the preset background noise value from each element in the feature matrix to obtain the feature matrix after removing the background noise.

[0157] S708, cluster the elements in the noise-removed feature matrix according to the number of sound sources to obtain multiple sound source sets, wherein each sound source set contains one or more elements.

[0158] S709 sorts the elements in each sound source set according to the order of time-frequency signal energy from small to large, removes the elements at the top of each sound source set by a preset proportion, and obtains the feature matrix after removing interference, including S7091, S7092 and S7093.

[0159] S7091, the clustered feature matrix is ​​put into the reshape() function to reshape it into a one-dimensional matrix, and then the one-dimensional matrix is ​​put into the np.histogram() function to perform histogram statistics, and the elements are arranged in order of energy from smallest to largest.

[0160] S7092 filters elements starting from the time-frequency points with low energy, removing a total of 40% of the time-frequency points to obtain a one-dimensional matrix after removing interference.

[0161] S7093 restores the one-dimensional matrix after removing interference to the feature matrix, thus obtaining the feature matrix after removing interference.

[0162] S710 uses the softmax function to transform the feature matrix after removing interference, resulting in a normalized feature matrix.

[0163] S711 generates interference-free time-spectrum data based on the normalized feature matrix. A short-time inverse Fourier transform is then performed on the interference-free time-spectrum data to obtain the enhanced audio data.

[0164] Based on the same inventive concept, this disclosure also provides a multi-channel audio data enhancement device, as shown in the following embodiments. Since the principle by which this device embodiment solves the problem is similar to that of the above-described method embodiments, the implementation of this device embodiment can refer to the implementation of the above-described method embodiments, and repeated details will not be elaborated further.

[0165] Figure 8 This diagram illustrates a multi-channel audio data enhancement processing apparatus according to an embodiment of the present disclosure, such as... Figure 8 As shown, the multi-channel audio data enhancement device 800 includes: an audio data acquisition module 801, a time-spectrum data generation module 802, a feature matrix generation module 803, a clustering module 804, an interference removal module 805, and an audio enhancement result output module 806.

[0166] The audio data acquisition module 801 acquires the audio data to be enhanced, which includes audio data collected from multiple channels. The time-spectrum data generation module 802 performs a short-time Fourier transform on the audio data of each channel to obtain the time-spectrum data corresponding to each channel. The feature matrix generation module 803 generates a feature matrix of the audio data based on the time-spectrum data of each channel, where each element in the feature matrix corresponds to a time-frequency point. The clustering module 804 clusters the elements in the feature matrix according to the number of sound sources to obtain multiple sound source sets, where each sound source set contains one or more elements. The interference removal module 805 sorts the elements in each sound source set according to the order of time-frequency signal energy from smallest to largest, removes a preset proportion of the elements in each sound source set, and obtains the feature matrix after interference removal. The audio enhancement result output module 806 performs a short-time inverse Fourier transform on the feature matrix after interference removal to obtain the enhanced audio data.

[0167] Furthermore, the aforementioned time-spectrum data generation module 802 can also be used to: normalize the audio data to be enhanced using formula (2).

[0168] In one embodiment of this disclosure, the feature matrix generation module 803 can also be used to: calculate the amplitude spectrum cross-correlation matrix between different channels based on the amplitude spectrum data; calculate the phase spectrum cross-correlation matrix between different channels based on the phase spectrum data; and generate a feature matrix based on the amplitude spectrum cross-correlation matrix and the phase spectrum cross-correlation matrix.

[0169] In specific implementation, the feature matrix generation module 803 can calculate the amplitude spectrum cross-correlation matrix between different channels using the above formula (7), calculate the phase spectrum cross-correlation matrix between different channels using the above formula (8), and calculate the feature matrix using the above formula (9).

[0170] In one embodiment of this disclosure, the clustering module 804 can also be used to: use the K-means algorithm to cluster the elements in the feature matrix according to the number of sound sources.

[0171] Furthermore, the clustering module 804 described above can also be used to: subtract a preset background noise value from each element in the feature matrix to obtain the feature matrix after removing the background noise.

[0172] In one embodiment of this disclosure, the audio enhancement result output module 806 can also be used to: generate interference-free time-spectrum data based on the feature matrix after interference removal; and perform a short-time inverse Fourier transform on the interference-free time-spectrum data to obtain enhanced audio data.

[0173] Furthermore, the aforementioned audio enhancement output module 806 can also be used to: transform the interference-removed feature matrix using the softmax function to obtain a normalized feature matrix; wherein, performing a short-time inverse Fourier transform on the interference-removed feature matrix to obtain enhanced audio data includes: performing a short-time inverse Fourier transform on the normalized feature matrix to obtain enhanced audio data. This disclosure normalizes the interference-removed feature matrix to ensure that the energy level of the original audio data remains unchanged, thereby achieving better audio processing results.

[0174] As can be seen from the above, the multi-channel audio data enhancement device provided in this embodiment, after acquiring the multi-channel audio data to be enhanced, performs a short-time Fourier transform on the audio data of each channel to obtain the time-spectrum data corresponding to each channel. Then, based on the time-spectrum data of each channel, a feature matrix of the audio data to be enhanced is generated. The elements in the feature matrix are clustered according to the number of sound sources, so that the elements belonging to each sound source are aggregated into a sound source set. The elements in each sound source set are sorted according to the time-frequency energy of each element. The elements at the top of the sorted preset proportion in each sound source set are removed to obtain the feature matrix after interference removal. Finally, a short-time inverse Fourier transform is performed on the feature matrix after interference removal to obtain the enhanced audio data. The clustering algorithm is used to obtain each sound source set and removes similar interference components in each sound source. Therefore, it can achieve better audio processing effect in noisy environments. In this embodiment, the feature matrix composed of the cross-correlation matrix between channels is used as the input of the clustering method, which ensures the efficiency of the audio processing technology and solves the problem in the prior art that the audio processing cannot be accurately performed due to the interference of noise in the environment.

[0175] Figure 9 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. Referring below... Figure 9 To describe an electronic device 900 according to such an embodiment of the present disclosure. Figure 9 The electronic device 900 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0176] like Figure 9 As shown, the electronic device 900 is presented in the form of a general-purpose computing device. The components of the electronic device 900 may include, but are not limited to: at least one processing unit 910, at least one storage unit 920, and a bus 930 connecting different system components (including the processing unit 910 and the storage unit 920).

[0177] The storage unit stores program code, which can be executed by the processing unit 910, causing the processing unit 910 to perform the steps described in the above-described section on the electronic prescription circulation processing method according to various exemplary embodiments of this disclosure. For example, the processing unit 910 can perform the following steps that can be performed in the above-described method embodiments: acquiring audio data to be enhanced, including data collected from multiple channels; performing a short-time Fourier transform on the audio data of each channel to obtain time-spectrum data corresponding to each channel; generating a feature matrix of audio data based on the time-spectrum data of each channel; clustering the elements in the feature matrix according to the number of sound sources to obtain multiple sound source sets containing one or more elements; sorting the elements in each sound source set according to the order of time-frequency signal energy from smallest to largest, removing a predetermined proportion of the elements in each sound source set to obtain a feature matrix after removing interference; performing a short-time inverse Fourier transform on the feature matrix after removing interference to obtain enhanced audio data.

[0178] Storage unit 920 may include readable media in the form of volatile storage units, such as random access memory (RAM) 9201 and / or cache memory 9202, and may further include read-only memory (ROM) 9203.

[0179] Storage unit 920 may also include a program / utility 9204 having a set (at least one) program module 9205, such program module 9205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0180] Bus 930 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0181] Electronic device 900 can also communicate with one or more external devices 940 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 900, and / or with any device that enables electronic device 900 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 950. Furthermore, electronic device 900 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 990. Network adapter 960 can communicate with other modules of electronic device 900 via bus 930. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0182] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, or network device, etc.) to execute the methods described above according to the embodiments of this disclosure.

[0183] In particular, according to embodiments of this disclosure, the process described above with reference to the flowchart can be implemented as a computer program product, which includes a computer program that, when executed by a processor, implements the above-described multi-channel audio data enhancement processing method.

[0184] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. Figure 10 This illustration shows a schematic diagram of a computer-readable storage medium according to an embodiment of the present disclosure, such as... Figure 10 As shown, the computer-readable storage medium stores a program product 1000 capable of implementing the methods described above in this disclosure. In some possible embodiments, various aspects of this disclosure may also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0185] More specific examples of computer-readable storage media in this disclosure may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0186] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0187] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing devices can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0188] The aforementioned computer-readable medium carries one or more programs, which, when executed by a device, cause the computer-readable medium to perform the following functions: acquire an image containing target handwriting; determine the skeleton information of the target handwriting using skeleton extraction technology, the skeleton information including straight line segments and / or curved line segments; generate path information using the skeleton information; and generate control instructions using the path information to control a robotic arm to imitate the target handwriting.

[0189] Those skilled in the art will understand that the above modules can be distributed in the device as described in the embodiments, or they can be modified accordingly and placed in one or more devices that are unique to this embodiment. The modules in the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0190] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0191] Exemplary embodiments of this disclosure have been specifically shown and described above. It should be understood that this disclosure is not limited to the detailed structures, arrangements, or implementations described herein; rather, this disclosure is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended claims.

[0192] Furthermore, the structures, proportions, sizes, etc. shown in the accompanying drawings of this disclosure are only used to complement the content disclosed in the specification, so as to enable those skilled in the art to understand and read them, and are not intended to limit the conditions under which this disclosure can be implemented. Therefore, they have no substantial technical significance. Any modification to the structure, change in the proportional relationship, or adjustment of the size should still fall within the scope of the technical content disclosed in this disclosure, provided that it does not affect the technical effects and purposes that this disclosure can produce.

Claims

1. A method of multi-channel audio data enhancement processing, characterized by, The method comprises the following steps: acquiring audio data to be enhanced, wherein the audio data comprises audio data collected by multiple channels; performing short-time Fourier transform on the audio data of each channel to obtain time-frequency spectrum data corresponding to each channel; generating a feature matrix of the audio data according to the time-frequency spectrum data of each channel, wherein each element in the feature matrix corresponds to a time-frequency point; clustering the elements in the feature matrix according to the number of sound sources to obtain multiple sound source sets, wherein each sound source set comprises one or more elements; sorting the elements in each sound source set in the order of time-frequency point signal energy from small to large, and removing the elements in each sound source set that are ranked in the front in a preset proportion to obtain a feature matrix after removing interference; performing short-time inverse Fourier transform on the feature matrix after removing interference to obtain enhanced audio data.

2. The multi-channel audio data enhancement processing method of claim 1, wherein, The clustering of the elements in the feature matrix according to the number of sound sources comprises: adopting a K-means algorithm to cluster the elements in the feature matrix according to the number of sound sources.

3. The multi-channel audio data enhancement processing method of claim 1, wherein, Before the clustering of the elements in the feature matrix according to the number of sound sources to obtain multiple sound source sets, the method further comprises: subtracting a preset background noise value from each element in the feature matrix to obtain a feature matrix after removing background noise; wherein the clustering of the elements in the feature matrix according to the number of sound sources to obtain multiple sound source sets comprises: clustering the elements in the feature matrix after removing background noise according to the number of sound sources to obtain multiple sound source sets.

4. The multi-channel audio data enhancement processing method of claim 1, wherein, The time-frequency spectrum data of each channel comprises amplitude spectrum data and phase spectrum data; wherein the generation of the feature matrix of the audio data according to the time-frequency spectrum data of each channel comprises: calculating an amplitude spectrum cross-correlation matrix between different channels according to the amplitude spectrum data; calculating a phase spectrum cross-correlation matrix between different channels according to the phase spectrum data; generating a feature matrix according to the amplitude spectrum cross-correlation matrix and the phase spectrum cross-correlation matrix.

5. The multi-channel audio data enhancement processing method of claim 4, wherein, The amplitude spectrum cross-correlation matrix between different channels is calculated by the following formula: r i,j = X a,i - X a,j The phase spectrum cross-correlation matrix between different channels is calculated by the following formula: s i,j = X p,i - X p,j The feature matrix is calculated by the following formula: where i and j represent channel order numbers, taking natural numbers; a represents amplitude; p represents phase; X a,i represents the amplitude spectrum matrix of the i-th channel; X a,j represents the amplitude spectrum matrix of the j-th channel; r i,j represents the amplitude spectrum cross-correlation matrix of the i-th and j-th channels; X p,i represents the phase spectrum matrix of the i-th channel; X p,j represents the phase spectrum matrix of the j-th channel; s i,j represents the phase spectrum cross-correlation matrix of the i-th and j-th channels; f represents a feature matrix; represents Hadamard product.

6. The multi-channel audio data enhancement processing method of claim 1, wherein, The short-time inverse Fourier transform on the feature matrix after removing interference to obtain enhanced audio data comprises: generating time-frequency spectrum data after removing interference according to the feature matrix after removing interference; performing short-time inverse Fourier transform on the time-frequency spectrum data after removing interference to obtain enhanced audio data.

7. The multi-channel audio data enhancement processing method according to any one of claims 1 to 6, characterized in that, Before the short-time inverse Fourier transform on the feature matrix after removing interference to obtain enhanced audio data, the method further comprises: transforming the feature matrix after removing interference by using a softmax function to obtain a normalized feature matrix; wherein the short-time inverse Fourier transform on the feature matrix after removing interference to obtain enhanced audio data comprises: performing short-time inverse Fourier transform on the normalized feature matrix to obtain enhanced audio data.

8. A multi-channel audio data enhancement apparatus, characterized by comprising: The method comprises the following steps: The audio data acquisition module is configured to acquire audio data to be enhanced, wherein the audio data comprises audio data collected by multiple channels; The time-frequency spectrum data generation module is configured to perform short-time Fourier transform on the audio data of each channel to obtain time-frequency spectrum data corresponding to each channel. The feature matrix generation module is configured to generate a feature matrix of the audio data according to the time-frequency spectrum data of each channel, wherein each element in the feature matrix corresponds to a time-frequency point. The clustering module is configured to cluster the elements in the feature matrix according to the number of sound sources to obtain multiple sound source sets, wherein each sound source set comprises one or more elements. The interference removal module is configured to sort the elements in each sound source set in order of time-frequency point signal energy from small to large, remove elements in each sound source set that are in a preset proportion at the front of the order, and obtain a feature matrix after interference removal. The audio enhancement result output module is configured to perform short-time inverse Fourier transform on the feature matrix after interference removal to obtain enhanced audio data.

9. An electronic device, comprising: The computer program is executed by the processor to implement the multi-channel audio data enhancement processing method of any one of claims 1-7. The computer program is executed by the processor to implement the multi-channel audio data enhancement processing method of any one of claims 1-7. ​ ​ ​ 10. A computer-readable storage medium having stored thereon a computer program, characterized in that, ​

Images