A data cleaning method and system

Abstract

The application belongs to the technical field of data processing, and particularly relates to a data cleaning method and system, which comprises the following steps: dividing the frequency domain signal of audio data into two categories; obtaining the complexity of the audio data according to the difference between the information entropy of the two categories of frequency domain signals and the information entropy of the audio data; performing empirical mode decomposition on the audio data to obtain IMF, and obtaining the noise property of the IMF according to the information entropy of the IMF, the number of extreme points and the complexity of the audio data; obtaining the reserved coefficient of the target IMF according to the energy difference of the target IMF, the noise property of the target IMF and the complexity of the audio data; multiplying each IMF with its reserved coefficient to obtain each reserved IMF, and accumulating each reserved IMF and the residual error to obtain de-noised audio data, thereby realizing audio data cleaning. The application performs data cleaning according to the characteristics of the audio data, and improves the data cleaning effect.

Description

Technical Field

[0001] This invention relates to the field of data processing technology. More specifically, this invention relates to a data cleaning method and system. Background Technology

[0002] In various fields such as finance, e-commerce, government and enterprise, education, and insurance, voice intelligence plays an important role in recognizing user intent and automatically analyzing user profiles. Typically, voice intelligence needs to be trained using voice data to enable it to possess the functions required in the corresponding field. Therefore, the realization of the functions of voice intelligence depends on training with high-quality voice data. In general, there is a certain amount of background noise in the voice data, which affects the training of voice intelligence. Therefore, it is necessary to clean the voice data.

[0003] In related technologies, for example, Chinese patent document with authorization announcement number CN117373471B discloses an audio data optimization and noise reduction method and system, including: acquiring the original audio signal; obtaining all extreme points of the original audio signal, obtaining the local feature value of each extreme point based on the amplitude change of data points in the local range of each extreme point; obtaining the target extreme point based on the difference in local features between each extreme point and adjacent extreme points; decomposing the original audio signal based on the target extreme point to obtain a first set of signals; and obtaining a second set of signals; obtaining the fluctuation degree of each component signal in the first set of signals and the second set of signals; obtaining the wavelet threshold of each component signal in the second set of signals based on the difference in the fluctuation degree of each component signal; denoising each component signal in the second set of signals based on the obtained wavelet thresholds; reconstructing the denoised component signals of the second set of signals to obtain the denoised audio signal.

[0004] In related technologies, target extreme points are selected by referencing the weights of extreme points and preset thresholds, and audio data is denoised by performing empirical mode decomposition based on the target extreme points. However, in continuous speech audio signals, the audio signal is affected by the constant changes in factors such as phoneme category and pronunciation intensity, resulting in a certain degree of instability in the speech audio signal. At the same time, different audio data have different characteristics, which makes it impossible to effectively suppress noise interference through data cleaning using related technologies. Summary of the Invention

[0005] To address the technical problem that the aforementioned technologies lack consideration for the characteristics of different audio signals, resulting in an inability to effectively suppress noise, the present invention provides solutions in the following aspects.

[0006] In a first aspect, the present invention provides a data cleaning method, comprising: performing Fourier transform on audio data to obtain frequency domain signals of the audio data; dividing the frequency domain signals of the audio data into two categories using a clustering algorithm; obtaining the complexity of the audio data based on the difference between the information entropy of the two categories of frequency domain signals and the information entropy of the audio data; performing empirical mode decomposition on the audio data to obtain several IMFs and residuals; taking any IMF as a target IMF; obtaining the noise level of the target IMF based on the information entropy of the target IMF and the number of extreme points in the target IMF; obtaining the energy difference based on the difference between a first energy value and a second energy value of the target IMF; obtaining the retention coefficient of the target IMF based on the energy difference of the target IMF, the noise level of the target IMF, and the complexity of the audio data; wherein the first energy value is the energy of the frequency domain signal of the target IMF, and the second energy value is the energy of the signal in the frequency domain signal of the target IMF between the minimum and maximum values ​​of the corresponding frequencies of human speech signals; multiplying each IMF by its retention coefficient to obtain each retained IMF; and accumulating each retained IMF and the residuals to obtain denoised audio data, thereby achieving audio data cleaning.

[0007] This invention can adaptively measure the signal complexity of each audio segment, combining the information entropy of the frequency and time domains, the noise and energy difference of each intrinsic mode function obtained from empirical mode decomposition, thereby assigning refined retention coefficients to each IMF, maximizing the preservation of speech components and reducing noise interference. Compared with existing technologies, this invention overcomes the deficiency of lacking consideration of the characteristics of audio data collected under different environments in data cleaning, improves the universality and robustness of data cleaning for audio data collected under different environments, and significantly improves the speech clarity of the denoised audio.

[0008] Preferably, the clustering algorithm is the K-means clustering algorithm.

[0009] Preferably, the complexity of the audio data is obtained as follows: the result of linearly normalizing the information entropy of the audio data time domain signal is denoted as the normalized time domain information entropy; the ratio between the difference between the information entropy of the two corresponding frequency domain signals of the audio data and the information entropy of the audio data frequency domain signal is denoted as the frequency domain information entropy ratio; and the product between the normalized time domain information entropy and the frequency domain information entropy ratio after negative correlation mapping is denoted as the complexity of the audio data.

[0010] This invention combines the temporal and frequency domain signal information entropy of audio data and further utilizes clustering algorithms to distinguish between suspected speech and suspected noise signals. This allows for precise measurement of the distinguishability between speech and noise signals in audio data, enabling the data cleaning system to adaptively adjust the denoising intensity for audio samples with different noise levels. For high-complexity audio, it avoids excessive denoising that could lead to the loss of speech details, while for low-complexity audio, it can implement more stringent noise suppression, thereby improving the overall denoising effect and speech quality.

[0011] Preferably, the noise level is obtained by: recording the ratio between the information entropy of the target IMF and the maximum value among the information entropies of all IMFs corresponding to the audio data as the first ratio; recording the ratio between the number of extreme points in the target IMF and the maximum number of extreme points in all IMFs corresponding to the audio data as the second ratio; and recording the product of the first ratio and the second ratio as the noise level of the target IMF.

[0012] This invention calculates the information entropy and the number of extreme points of the intrinsic modulus function of each audio data, and quantifies its noise accordingly. It can accurately identify the noise-dominated IMF, while avoiding the speech distortion caused by excessive removal of IMFs containing speech signals due to the indiscriminate elimination of IMFs, thus maintaining the quality of speech signals in audio data.

[0013] Preferably, the energy difference is obtained as follows: the difference between the second frequency and the first frequency is recorded as the frequency difference; the ratio between the first energy value of the target IMF and the maximum frequency value of the target IMF frequency domain signal is recorded as the first average energy value; the ratio between the second energy value of the target IMF and the frequency difference is recorded as the second average energy value; and the normalized result of the difference between the first average energy value and the second average energy value is recorded as the energy difference of the target IMF, wherein the first frequency is 300Hz and the second frequency is 3400Hz.

[0014] This invention compares the overall frequency domain energy of each IMF with the energy of the speech segment, which can accurately determine the energy distribution ratio of speech and noise in the IMF. The energy difference can dynamically reflect the strength of noise in different IMFs, thereby achieving more effective separation of noise and providing a reliable basis for the subsequent calculation of retention coefficients, thus improving the robustness of data cleaning in complex noise environments.

[0015] Preferably, the retention coefficient is obtained as follows: the negative correlation mapping value of the product between the energy difference of the target IMF and the noise normalization value of the target IMF is recorded as the first retention degree, and the product between the first retention degree and the complexity of the audio data corresponding to the target IMF is recorded as the retention coefficient of the target IMF, wherein the noise normalization value of the target IMF is the ratio between the noise of the target IMF and the sum of the noise of all IMFs corresponding to the audio data.

[0016] Preferably, the retention coefficient is obtained as follows: the negative correlation mapping value of the product between the energy difference of the target IMF and the noise normalization value of the target IMF is recorded as the first retention degree, and the product between the first retention degree and the complexity of the audio data corresponding to the target IMF is recorded as the retention coefficient of the target IMF, wherein the noise normalization value of the target IMF is the ratio between the noise of the target IMF and the maximum value among all the noise values ​​of the IMFs corresponding to the audio data.

[0017] This invention comprehensively utilizes the noise characteristics and energy difference of IMFs and the overall complexity of the audio to obtain retention coefficients, thereby denoising audio data according to the characteristics of different IMFs. This avoids the blind retention or removal of all IMFs by a uniform strategy, thus balancing the integrity of the speech signal in the data with the noise reduction effect, resulting in better data cleaning.

[0018] Preferably, the step of multiplying each IMF with its retention coefficient to obtain each retained IMF includes: multiplying each value in each IMF with its retention coefficient to obtain each value in each retained IMF.

[0019] Preferably, the step of accumulating each retained IMF and the residual to obtain denoised frequency data includes: adding each value in each retained IMF to each value in the residual in tandem to obtain each value in the denoised frequency data.

[0020] Secondly, the present invention provides a data cleaning system, including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the above-mentioned data cleaning method is implemented.

[0021] By adopting the above technical solution, a data cleaning method is generated into a computer program and stored in a memory for loading and execution by a processor. This allows for the creation of a terminal device based on the memory and processor, facilitating its use.

[0022] The beneficial effects of this invention are as follows: By analyzing the time-domain and frequency-domain signals of audio data, this invention utilizes information entropy and clustering to initially separate signal types and obtain complexity, capturing the unique characteristics of audio data, which helps guide denoising strategies; based on the information entropy and number of extreme points of each IMF, it can precisely identify the noise-dominated components in each IMF, ensuring the integrity of the speech signal is preserved during denoising, laying the foundation for subsequent processing; based on the total energy of each IMF and the energy of the speech frequency range, it obtains the energy difference, allowing data cleaning to focus on specific frequency bands of the speech signal, ensuring that the denoising process prioritizes the preservation of frequencies related to the speech signal, thereby improving the accuracy and efficiency of denoising; based on the noise level, energy difference, and audio complexity, it obtains the retention coefficient, which can balance the needs of noise suppression and speech preservation, ensuring optimal speech preservation and noise reduction in different noise environments. Audio denoising through the retention coefficient can ensure the preservation of speech information while reducing noise interference, significantly improving the quality of audio data, thereby improving the overall quality of the dataset and enhancing the training effect of the speech agent. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating a data cleaning method according to the present invention;

[0024] Figure 2 It is a schematic diagram showing the comparison between the time-domain and frequency-domain signals of audio data;

[0025] Figure 3 This is a schematic diagram illustrating the results of clustering frequency domain signals. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0028] This invention discloses a data cleaning method, referring to... Figure 1 This includes steps S1-S6:

[0029] S1. Obtain the audio dataset and calculate the complexity of each audio data point based on its frequency domain and time domain signals.

[0030] It should be noted that audio data is a series of waveforms that change over time, which is a time-domain signal. Although time-domain signals can reflect the instantaneous changes, rhythm, and pauses of audio data, it is not easy to analyze the difference between speech signals and noise signals in the data. In the frequency domain, speech signals are often concentrated in certain specific frequency bands, while noise signals usually cover a wider frequency range. There is a certain difference between speech signals and noise signals in the frequency domain. In order to measure the difference between speech signals and noise signals, this invention obtains the complexity of each audio data point based on the frequency domain signal and time domain signal corresponding to each audio data point in the audio dataset.

[0031] Specifically, the audio dataset required for training the speech agent is obtained, the information entropy of each audio data (time domain signal) is obtained, Fourier transform is performed on the audio data to obtain the frequency domain signal of the audio data, the information entropy of the frequency domain signal is obtained, and the frequency domain signal of the audio data is divided into two categories by clustering algorithm using the amplitude of the audio data in the frequency domain signal. The category with a large frequency range is suspected noise signal, and the category with a small frequency range is suspected speech signal. The information entropy of the frequency domain signal of each category is obtained, and the difference between the information entropy of the suspected speech signal and the information entropy of the suspected noise signal is obtained. The complexity of the audio data is obtained based on this difference and the information entropy of the audio data.

[0032] It should be further explained that in the frequency domain signal of audio data, there is a certain difference between the speech signal and the noise signal in terms of frequency. Therefore, clustering algorithms can be used to initially divide the frequency domain signal of audio data into suspected speech signals and suspected noise signals. However, there is also a certain overlap between the speech signal and the noise signal in the frequency domain, which leads to inaccurate classification of the signal by the clustering algorithm. If the suspected noise signal is directly removed, it may cause partial loss of the speech signal, affecting the quality of the speech signal in the audio data. Therefore, this invention obtains the complexity of the audio data based on the difference between the suspected speech signal and the suspected noise signal, and performs audio data denoising based on the complexity.

[0033] Preferably, the clustering algorithm is the K-means clustering algorithm.

[0034] For example, Figure 2 This is a comparison graph of the time-domain and frequency-domain signals of audio data. The graph shows the time-domain and frequency-domain signals of a segment of audio data. In the time-domain signal, the horizontal axis represents the acquisition time corresponding to each sample in the signal, and in the frequency-domain signal, the horizontal axis represents the frequency of the signal. Figure 3 The image shows the results of clustering the frequency domain signal. Gray data points in the image represent suspected speech signals, while black data points represent suspected noise signals.

[0035] Preferably, the complexity of the audio data satisfies the following relationship:

[0036] ;

[0037] In the formula, For the first The complexity of audio data, For the first Information entropy of audio data in the time domain, For the first Information entropy of the frequency domain signal of the audio data. For the first The difference between the information entropy of the suspected speech signal and the information entropy of the suspected noise signal corresponding to each audio data point. It is an exponential function with the natural constant as its base. It is a linear normalization function.

[0038] in, The larger the number, the more likely it is to be the first. The higher the uncertainty of each audio data point, the more likely it is to contain rapidly changing and complex voice dialogues. The more complex the audio data, the more difficult it is to distinguish between the speech signal and the noise signal in the audio data. The greater the complexity of the audio data; The smaller the number, the higher the value. The higher the certainty of each audio data point, the more likely it is to contain a single tone or a monotone speech dialogue, and the easier it is to distinguish the speech signal from the noise signal in the audio data. The lower the complexity of each audio data point.

[0039] Represents the first The difference in information entropy between the two types of frequency domain signals in an audio data set. A larger value indicates a greater difference in the information content between the suspected speech signal and the suspected noise signal, making it easier to distinguish between speech and noise signals in the audio data. The lower the complexity of the audio data, the smaller the difference between the information content of the suspected speech signal and the suspected noise signal. This indicates a greater similarity in features between the speech and noise data, a more complex structure in the speech data, and greater difficulty in distinguishing between speech and noise signals in the audio data. The higher the complexity of the audio data, the more complex it becomes; for ease of calculation, the formula is modified by... Divided by the Information entropy of audio data frequency domain signal Adjust the numerical range and... use The function is normalized for negative correlation.

[0040] S2. Obtain the IMF of each speech data through empirical mode decomposition. Based on the information entropy and number of extreme points of the IMF of each audio data, obtain the noise level of each IMF of each audio data.

[0041] It should be noted that empirical mode decomposition can decompose audio data into multiple intrinsic mode functions (IMFs). During the decomposition process, noise signals are also decomposed into the IMFs. Noise removal of audio data can be achieved by removing the IMFs containing noise signals and then recombining the remaining IMFs. However, noise signals are not concentrated in specific IMFs. If the removed IMFs contain more speech signals, it may affect the quality of the audio data. In order to better remove noise from audio data, this invention obtains the noise characteristics of each intrinsic mode function based on the information entropy and the number of extreme points of each intrinsic mode function.

[0042] Specifically, empirical mode decomposition (EMD) is performed on each audio data (time-domain signal) to obtain several intrinsic mode functions (IMFs) and residuals for each audio data. The information entropy of each IMF and the number of extreme points in each IMF are obtained. Based on the information entropy and the number of extreme points in each IMF, the noise level of each intrinsic mode function (IMF) is obtained. During EMD, if the energy of the residual is less than an energy threshold, the decomposition is stopped. In this embodiment, the energy threshold is 0.05, but implementers can set the energy threshold according to actual conditions. It should be noted that the residual is also a signal, and the energy of the residual is calculated in the same way as the signal energy. The calculation of signal energy is a well-known technique and will not be elaborated here.

[0043] Preferably, the noise characteristic satisfies the following relationship:

[0044] ;

[0045] In the formula, For the first The first audio data The noise characteristics of an IMF For the first The first audio data Information entropy of an IMF For the first Each audio data point corresponds to the maximum value of the information entropy across all IMFs. For the first The first audio data The number of extreme points contained within an IMF. For the first The maximum number of extreme points corresponding to each audio data point across all IMFs.

[0046] in, Represents the first The first audio data The degree of disorder in an IMF (Instantaneous Multifunction Array) is determined by the fact that noise signals exhibit higher randomness and irregularity in both the time and frequency domains. Therefore, a larger value indicates that the IMF possesses more characteristics of a noise signal, and its noise level is higher; conversely, a smaller value indicates that the IMF does not possess the characteristics of a noise signal, and its noise level is lower. Similarly, since noise signals exhibit rapid and irregular oscillations in the time domain, therefore... The larger it is, the more it means the first The first audio data The more frequently an IMF fluctuates in the time domain, the higher the proportion of noise components in that IMF, and the greater the noise level of that IMF. The smaller, the more likely it is to be the first The first audio data The smoother the fluctuation of an IMF in the time domain, the lower the proportion of noise components in the IMF, and the less noisy the IMF is.

[0047] S3. Obtain the energy difference of each IMF based on the energy of the frequency domain signal of the IMF of each audio data.

[0048] Specifically, each IMF is converted into a frequency domain signal by Fourier transform, and the energy of each IMF frequency domain signal is obtained. This energy is used as the first energy value. The energy of the signal between the first frequency and the second frequency in the IMF frequency domain signal is obtained, and this energy is used as the second energy value. The retention coefficient is obtained based on the first energy value, the second energy value and the noise of the IMF. In this embodiment, the first frequency is 300Hz and the second frequency is 3400Hz. The implementer can determine the values ​​of the first frequency and the second frequency according to the actual situation.

[0049] It should be noted that the frequency domain signal of speech generated by human dialogue is usually between a first frequency and a second frequency. The first energy value represents the overall energy of the IMF frequency domain signal, and the second energy value represents the energy of the speech signal in the IMF. The larger the difference between the first energy value and the second energy value, the less speech signal and the more noise signal in the IMF. The smaller the difference between the first energy value and the second energy value, the more speech signal and the less noise signal in the IMF. Therefore, this invention obtains the energy difference of each IMF based on the difference between the first energy value and the second energy value.

[0050] Preferably, the energy difference satisfies the following relationship:

[0051] ;

[0052] In the formula, For the first The first audio data The energy difference between IMFs For the first The first audio data The first energy value of an IMF For the first The first audio data The second energy value of the IMF, The difference between the second frequency and the first frequency. For the first The first audio data The maximum frequency value of an IMF frequency domain signal It is a linear normalization function.

[0053] in, Represents the first The first audio data The average energy value of each frequency of an IMF Represents the first The first audio data The average energy value of each IMF at each frequency between the first and second frequencies. This represents the difference between the average energy value of each frequency in the IMF and the average energy value of each frequency between the first and second frequencies. The larger this value is, the greater the energy difference in the IMF; the smaller this value is, the smaller the energy difference in the IMF.

[0054] S4. Obtain the retention factor of each IMF based on the noise level of each IMF, the energy difference of each IMF, and the complexity of the audio data.

[0055] It should be noted that since the proportion of speech information to noise information contained in different IMFs is different, uniformly removing or retaining IMFs may lead to the accidental deletion of speech information or the retention of noise. Therefore, this invention obtains the retention coefficient of each IMF based on the noise level of each IMF, the energy difference of each IMF, and the complexity of the audio data, so that the speech information in the audio data is retained as much as possible and the noise is reduced as much as possible.

[0056] Preferably, the retention coefficient satisfies the following relationship:

[0057] ;

[0058] In the formula, For the first The first audio data The retention factor of each IMF, For the first The first audio data The energy difference between IMFs For the first The complexity of audio data, For the first The first audio data The noise characteristics of an IMF For the first The number of IMFs corresponding to each audio data point It is an exponential function with the natural constant as its base. In this embodiment, the parameters are customized. The implementers can choose according to the actual situation. The value of .

[0059] in, A larger value indicates a smaller proportion of speech signals and a larger proportion of noise signals in the IMF. The first audio data The smaller the retention factor of an IMF should be; The smaller the value, the greater the proportion of speech signal and the smaller the proportion of noise signal in the IMF. The first audio data The larger the retention factor of an IMF should be. Represents the first The first audio data The relative magnitude of the noise of the first IMF; the larger the value, the more significant the noise. The first audio data The more noise a single IMF may contain, the more it should be reduced in the audio data. The first IMF reservation, the first The first audio data The smaller the retention factor of the first IMF, the better; the smaller the value, the better. The first audio data The less noise a given IMF may contain, the more it should be increased. The first IMF reservation, the first The first audio data The larger the retention factor of each IMF, the better. Since audio data obtained under different acquisition environments have different characteristics, the complexity... The higher the audio data quality, the more difficult it is to distinguish between speech and noise signals. To avoid incorrectly removing IMFs containing more speech data, the retention factor of the IMF corresponding to the audio data should be larger; regarding complexity... The lower the audio data, the smaller the retention factor of the corresponding IMF should be.

[0060] S5. Denoise the audio data based on the retention coefficient of the IMF corresponding to each audio data.

[0061] Specifically, the IMF is multiplied by the retention coefficient corresponding to the IMF to obtain the retained IMF, the retained IMF corresponding to the audio data is accumulated, and the accumulated result of the retained IMF is added to the residual to obtain the denoised audio data; where the residual is the residual obtained by performing empirical mode decomposition on the audio data.

[0062] It should be noted that both the IMF and the residuals are time-series data. Multiplying the IMF by its retention factor actually means multiplying each number in the IMF by its retention factor. The process of adding the accumulated result of the retained IMF to the residuals is actually adding the accumulated result of the retained IMF to the residuals in pairs. That is, the first number in the accumulated result is added to the first number in the residual to get the first number in the denoised frequency data, the second number in the accumulated result is added to the second number in the residual to get the second number in the denoised frequency data, and so on.

[0063] Specifically, the denoised frequency data satisfies the following relationship:

[0064] ;

[0065] In the formula, For the first The denoising results of the audio data. For the first The first audio data One IMF, For the first The first audio data The retention factor of each IMF, For the first The number of IMFs corresponding to each audio data point For the first The residuals of empirical mode decomposition of audio data.

[0066] S6. Train the voice agent using the denoised audio data.

[0067] Specifically, in the process of training a voice agent in a specific field, such as finance or education, the audio data in the audio dataset of that field is denoised, and the denoised audio data is used to train the voice agent in that field.

[0068] This invention also discloses a data cleaning system, including a processor and a memory. The memory stores computer program instructions, which, when executed by the processor, implement a data cleaning method according to the invention.

[0069] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.

Claims

1. A data cleaning method, characterized in that, include: The audio data is subjected to Fourier transform to obtain the frequency domain signal of the audio data. The frequency domain signal of the audio data is divided into two categories by clustering algorithm. The complexity of the audio data is obtained based on the difference between the information entropy of the two categories of frequency domain signals and the information entropy of the audio data. Empirical mode decomposition is performed on the audio data to obtain several IMFs and residuals. Any IMF is taken as the target IMF, and the noise of the target IMF is obtained based on the information entropy of the target IMF and the number of extreme points in the target IMF. The energy difference is obtained based on the difference between the first and second energy values ​​of the target IMF; the retention factor of the target IMF is obtained based on the energy difference of the target IMF, the noise level of the target IMF, and the complexity of the audio data. The first energy value is the energy of the target IMF frequency domain signal, and the second energy value is the energy of the signal in the target IMF frequency domain signal between the minimum and maximum values ​​of the corresponding frequency of the human speech signal. Each IMF is multiplied by its retention coefficient to obtain each retained IMF. The retained IMFs and the residuals are then summed to obtain the denoised audio data, thus achieving audio data cleaning. The complexity of audio data is obtained as follows: the result of linearly normalizing the information entropy of the audio data time domain signal is denoted as the normalized time domain information entropy; the ratio between the difference between the information entropy of the two corresponding frequency domain signals of the audio data and the information entropy of the audio data frequency domain signal is denoted as the frequency domain information entropy ratio; and the product between the normalized time domain information entropy and the frequency domain information entropy ratio after negative correlation mapping is denoted as the complexity of the audio data. The retention coefficient is obtained as follows: the negative correlation mapping value of the product between the energy difference of the target IMF and the noise normalization value of the target IMF is recorded as the first retention level, and the product between the first retention level and the complexity of the audio data corresponding to the target IMF is recorded as the retention coefficient of the target IMF. The noise normalization value of the target IMF is the ratio between the noise of the target IMF and the sum of the noise of all IMFs corresponding to the audio data.

2. The data cleaning method according to claim 1, characterized in that, The clustering algorithm is the K-means clustering algorithm.

3. The data cleaning method according to claim 1, characterized in that, The method for obtaining the noise level is as follows: The first ratio is defined as the ratio between the information entropy of the target IMF and the maximum value among all IMFs corresponding to the audio data. The second ratio is defined as the ratio between the number of extreme points in the target IMF and the maximum number of extreme points in all IMFs corresponding to the audio data. The product of the first ratio and the second ratio is defined as the noise level of the target IMF.

4. The data cleaning method according to claim 1, characterized in that, The energy difference is obtained in the following way: The difference between the second frequency and the first frequency is denoted as the frequency difference. The ratio between the first energy value of the target IMF and the maximum frequency value of the target IMF frequency domain signal is denoted as the first average energy value. The ratio between the second energy value of the target IMF and the frequency difference is denoted as the second average energy value. The normalized result of the difference between the first average energy value and the second average energy value is denoted as the energy difference of the target IMF. The first frequency is 300Hz and the second frequency is 3400Hz.

5. The data cleaning method according to claim 1, characterized in that, The step of multiplying each IMF with its retention factor to obtain each retained IMF includes: multiplying each value in each IMF with its retention factor to obtain each value in each retained IMF.

6. The data cleaning method according to claim 1, characterized in that, The step of accumulating each retained IMF and the residual to obtain denoised frequency data includes: adding each value in each retained IMF to each value in the residual in tandem to obtain each value in the denoised frequency data.

7. A data cleaning system, characterized in that, include: A processor and a memory, the memory storing computer program instructions that, when executed by the processor, implement a data cleaning method according to any one of claims 1-6.

Images