Music noise sample filtering methods, electronic devices and storage media

Abstract

This application discloses a method, electronic device, and storage medium for filtering music noise samples. The method includes: acquiring a music sample to be processed; extracting the representational features of the music sample in the time domain, the frequency spectrum, and the latent space; performing weighted fusion processing on the representational features in the time domain, the frequency spectrum, and the latent space to determine a target score; if the target score meets a preset condition, the music sample to be processed is music noise, and the music sample is then filtered. This solution solves the technical problem that existing binary classification models cannot accurately filter noise samples in music scenes, achieving a more accurate and flexible filtering effect for noise in music scenes.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to a method for filtering music noise samples, an electronic device, and a storage medium. Background Technology

[0002] Noise filtering is a crucial step in data processing for any audio task. In music data processing, the cleanliness and accuracy of filtering out music-irrelevant noise samples directly impacts the final performance of the trained model. Inaccurate noise filtering can mislead model training and affect the true distribution of the music data. Therefore, accurately capturing and filtering the characteristics of music noise is an essential part of preprocessing.

[0003] Considering that the noise in music scenes is more complex than that in ordinary tasks, the non-musical noise samples include not only common noises such as scene noise and noisy human voices, but also single-tone noise that is easily confused with music and instrument noise without any rhythm.

[0004] Current noise filtering methods generally model and filter noise in the same way as ordinary noise analysis, without making changes for music scenarios. They mainly train noise classification models based on specific scenarios. In music scenarios, they train binary classification models to distinguish between music and non-music noise. However, noise is diverse, and the spatial distribution of binary noise samples cannot fully encompass all noise characteristics in real scenarios. For example, noise samples need to cover scene noise, but it is impossible to collect noise samples from all scenarios. Therefore, for music scenarios, binary classification models cannot accurately filter noise samples.

[0005] The background description is provided for the purpose of understanding the relevant technologies in this field and is not intended as an admission of prior art. Summary of the Invention

[0006] Therefore, embodiments of the present invention aim to provide a method, electronic device, and storage medium for filtering music noise samples, so as to accurately filter noise samples in music scenes.

[0007] In a first aspect, a method for filtering music noise samples is provided, comprising:

[0008] Obtain the music sample to be processed;

[0009] Extract the representation features of the music sample to be processed in the time domain, the spectral space, and the latent space;

[0010] The target score is determined by weighted fusion of the representation features in the time domain, the representation features in the spectral space, and the representation features in the latent space.

[0011] If the target score meets the preset conditions, then the music sample to be processed is music noise, and the music sample to be processed is filtered.

[0012] In one implementation, the representation features in the time domain, the representation features in the spectral space, and the representation features in the latent space are weighted and fused to determine the target score, including:

[0013] Calculate the first score of the representation features in the time domain;

[0014] Calculate the second score of the representation features in the spectral space;

[0015] Calculate the third score of the representation features in the latent space;

[0016] Based on the pre-set weight ratios, the first score, the second score, and the third score are weighted and summed to obtain the target score.

[0017] If the target score is determined to meet the preset conditions, then the music sample to be processed is determined to be music noise.

[0018] In one implementation, when the representation feature in the time domain space has a zero-crossing rate, calculating the first score of the representation feature in the time domain space includes:

[0019] Extract the zero-crossing rate of the music sample to be processed;

[0020] The zero-crossing rate is compared with a preset zero-crossing threshold;

[0021] When the zero-crossing rate is greater than the zero-crossing threshold, the first zero-crossing value is used as the first score; when the zero-crossing rate is not greater than the zero-crossing threshold, the second zero-crossing value is used as the first score. The first zero-crossing value represents a high noise probability, and the second zero-crossing value represents a low noise probability.

[0022] In one implementation, the zero-crossing threshold is set as follows:

[0023] Obtain music data sample sets and noise data sample sets;

[0024] The mean zero-crossing rate of the data samples in the music data sample set is determined as the first mean;

[0025] The mean zero-crossing rate of the data samples in the noise data sample set is determined as the second mean;

[0026] The sum of the first mean and the second mean is taken as the zero-crossing threshold.

[0027] In one implementation, when the representation feature in the spectral space is the spectral centroid variance, calculating the second score of the representation feature in the spectral space includes:

[0028] Extract the spectral centroid variance of the music sample to be processed;

[0029] The spectral centroid variance is compared with a preset centroid threshold.

[0030] If the centroid variance of the spectrum is greater than a preset centroid variance threshold, the first centroid value is used as the second score; if the centroid variance of the spectrum is not greater than the centroid variance threshold, the second centroid value is used as the second score. The first centroid value represents a low noise probability, and the second centroid value represents a high noise probability.

[0031] In one implementation, the centroid variance threshold is set as follows:

[0032] Obtain music data sample sets and noise data sample sets;

[0033] The music data sample set is divided into multiple frames, and the spectral centroid point of each frame is determined to obtain the spectral centroid sequence corresponding to each music data sample. Based on the spectral centroid sequence corresponding to each music data sample, the spectral centroid variance of each music data sample is calculated.

[0034] The mean of the centroid variance of the spectrum of each music sample in the music data sample set is obtained and used as the third mean.

[0035] The noise data sample set is divided into multiple frames, and the spectral centroid point of each frame is determined to obtain the spectral centroid sequence corresponding to each noise data sample. Based on the spectral centroid sequence corresponding to each noise data sample, the spectral centroid variance of each noise data sample is calculated.

[0036] The mean of the centroid variance of the spectrum of each noise sample in the noise data sample set is obtained as the fourth mean.

[0037] The sum of the third mean and the fourth mean is taken as the centroid variance threshold.

[0038] In one implementation, when the representation features in the latent space are the output of the embedding layer, the third score of the representation features in the latent space is calculated, including:

[0039] Extract the Mel frequency cepstral coefficient features of the music sample to be processed;

[0040] The Mel frequency cepstral coefficient features are input into a pre-established autoencoder network to obtain the embedding layer output of the music sample to be processed.

[0041] Calculate the distance between the embedding layer output of the music sample to be processed and the mean output of the music embedding layer and the mean output of the noise embedding layer;

[0042] When the distance between the embedding layer output of the music sample to be processed and the average output of the music embedding layer is close, the first distance value is used as the third score. When the distance between the embedding layer output of the music sample to be processed and the average output of the noise embedding layer is close, the second distance value is used as the third score. The first distance value represents a low noise probability, and the second distance value represents a high noise probability.

[0043] In one implementation, the output mean of the music embedding layer and the output mean of the noise embedding layer are set as follows:

[0044] Obtain music data sample sets and noise data sample sets;

[0045] Mel frequency cepstral coefficient features of each music data sample in the music data sample set are extracted. The Mel frequency cepstral coefficient features of each music data sample are input into a pre-established autoencoder network to obtain the embedding layer output of each music data sample. The mean of the embedding layer output of each music data sample is calculated as the mean of the music embedding layer output.

[0046] The Mel frequency cepstral coefficient features of each noise data sample in the noise data sample set are extracted. The Mel frequency cepstral coefficient features of each noise data sample are input into a pre-established autoencoder network to obtain the embedding layer output of each noise data sample. The mean of the embedding layer output of each noise data sample is calculated as the mean of the noise embedding layer output.

[0047] In a second aspect, a music noise sample filtering device is provided, comprising:

[0048] The acquisition module is used to acquire music samples to be processed.

[0049] The extraction module is used to extract the representation features of the music sample to be processed in the time domain, the spectral space, and the latent space.

[0050] The determination module is used to perform weighted fusion processing on the representation features in the time domain, the representation features in the spectral space, and the representation features in the latent space to determine the target score;

[0051] The filtering module is used to filter the music sample to be processed when the music sample to be processed is music noise.

[0052] In a third aspect, an electronic device is provided, comprising: a processor and a memory storing a computer program, the processor being configured to perform the processing method described in any embodiment when running the computer program.

[0053] In a fourth aspect, a storage medium is provided that stores a computer program configured to execute the processing method described in any embodiment when run.

[0054] This invention proposes an improved method for filtering music noise samples. Starting from the characteristics of noise samples, it employs a variety of effective features and does not rely on complex and diverse noise samples to construct a feature space. Therefore, it is more efficient and faster to implement. Compared with binary classification noise filtering methods, this application uses the differences in the characteristics of noise and music in the spectral and temporal domains, as well as their distribution in the potential feature space, to perform weighted fusion of features, thereby making noise filtering in music scenes more accurate and more flexible.

[0055] Optional features and other effects of embodiments of the present invention are described in part below, and in part will be apparent from reading this document. Attached Figure Description

[0056] The embodiments of the present invention will be described in detail with reference to the accompanying drawings. The elements shown are not limited to the scale shown in the drawings, and the same or similar reference numerals in the drawings denote the same or similar elements, wherein:

[0057] Figure 1 An exemplary flowchart of a music noise sample filtering method according to an embodiment of the present invention is shown;

[0058] Figure 2 A logical schematic diagram illustrating the implementation of a music noise sample filtering method based on multi-feature fusion according to an embodiment of the present invention is shown.

[0059] Figure 3 A schematic diagram of the structure of a music noise sample filtering device according to an embodiment of the present invention is shown;

[0060] Figure 4 A schematic diagram of an exemplary hardware structure of a mobile terminal capable of implementing the method according to an embodiment of the present invention is shown;

[0061] Figure 5 A schematic diagram of an exemplary operating system structure of a mobile terminal capable of implementing the methods according to embodiments of the present invention is shown;

[0062] Figure 6 A schematic diagram of an exemplary operating system structure for a mobile terminal capable of implementing methods according to embodiments of the present invention is shown. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0064] Figure 1 This is a flowchart of one embodiment of the music noise sample filtering method provided in this application. Although this application provides method operation steps or apparatus structures as shown in the following embodiments or figures, more or fewer operation steps or module units may be included in the method or apparatus based on conventional or non-inventive effort. In steps or structures where there is no logically necessary causal relationship, the execution order of these steps or the module structure of the apparatus is not limited to the execution order or module structure described in the embodiments and figures of this application. When the method or module structure is applied in actual devices or terminal products, it can be executed sequentially or in parallel according to the method or module structure shown in the embodiments or figures (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed processing environment).

[0065] Specifically, such as Figure 1 As shown, the above-mentioned music noise sample filtering method may include the following steps:

[0066] Step 101: Obtain the music sample to be processed;

[0067] In this example, the music sample to be processed may be a music sample or a noise sample. If it is a noise sample, then it is composed of various non-musical noises, which may include, but are not limited to, at least one of the following: scene noise, irrelevant human voices, instrument noise, etc.

[0068] Step 102: Extract the representation features of the music sample to be processed in the time domain, the spectral space, and the latent space;

[0069] The latent space is the feature space in which data is represented in a lower-dimensional way, i.e., a low-dimensional feature space. This representation captures the high-level abstract features and latent structure of the data, and can be seen as a dimensionality reduction representation of the data, in which the location and distribution of data points reflect the inherent relationships of the data.

[0070] The generation methods of the latent space include, but are not limited to, autoencoders, variational autoencoders (VAEs), and generative adversarial networks (GANs). Those skilled in the art can choose the generation method of the latent space according to their needs. The generation methods listed above are only examples and are not intended to limit the technical solutions described in this invention.

[0071] Step 103: Perform weighted fusion processing on the representation features in the time domain, the representation features in the spectral space, and the representation features in the latent space to determine the target score;

[0072] Step 104: If the target score meets the preset conditions, the music sample to be processed is music noise, and the music sample to be processed is filtered.

[0073] In the example above, starting from the characteristics of the noise samples, a variety of effective features were adopted. The feature space is not constructed based on complex and diverse noise samples, so it is more efficient and faster to implement. Compared with the binary classification noise filtering method, this application uses the differences in the characteristics of noise and music in the spectral space and temporal space, as well as their distribution in the potential feature space, to perform weighted fusion of features, thereby making the noise filtering in music scenes more accurate and more flexible.

[0074] In practical implementation, a weighted fusion method can be used, specifically, the representation features in the time domain, the representation features in the spectral space, and the representation features in the latent space are weighted and fused to determine the target score, which may include:

[0075] S1: Calculate the first score of the representation features in the time domain space;

[0076] For example, taking the zero-crossing rate as a representation feature in the time domain, calculating the first score of the representation feature in the time domain may include: extracting the zero-crossing rate of the music sample to be processed; comparing the zero-crossing rate with a preset zero-crossing threshold; if the zero-crossing rate is greater than the zero-crossing threshold, taking the first zero-crossing value as the first score, and if the zero-crossing rate is not greater than the zero-crossing threshold, taking the second zero-crossing value as the first score, wherein the first zero-crossing value represents a high probability of noise, and the second zero-crossing value represents a low probability of noise.

[0077] The zero-crossing threshold can be set as follows: acquire a music data sample set and a noise data sample set; determine the mean zero-crossing rate of the data samples in the music data sample set as the first mean; determine the mean zero-crossing rate of the data samples in the noise data sample set as the second mean; and take half of the sum of the first mean and the second mean as the zero-crossing threshold.

[0078] For example, assuming the average zero-crossing rate of all music samples in the music sample dataset is m1, and the average zero-crossing rate of all noise samples in the noise sample dataset is m2, then the zero-crossing threshold can be set as a = (m1 + m2) / 2. During the prediction phase, if the zero-crossing rate of the music sample to be processed is greater than a, the score can be set to -1; if the zero-crossing rate is less than a, the score can be set to 1.

[0079] S2: Calculate the second score of the representation features in the spectral space;

[0080] For example, taking the spectral centroid variance as a representation feature in the spectral space as an example, calculating the second score of the representation feature in the spectral space can include:

[0081] S2-1: Extract the centroid variance of the spectrum of the music sample to be processed;

[0082] S2-2: Compare the spectral centroid variance with a preset centroid threshold;

[0083] S2-3: When the centroid variance of the spectrum is greater than a preset centroid variance threshold, the first centroid value is used as the second score; when the centroid variance of the spectrum is not greater than the centroid variance threshold, the second centroid value is used as the second score. The first centroid value represents a low noise probability, and the second centroid value represents a high noise probability.

[0084] The centroid variance threshold can be set as follows: Acquire a music data sample set and a noise data sample set; divide each music sample data in the music data sample set into multiple frames, determine the spectral centroid point of each frame to obtain the spectral centroid sequence corresponding to each music sample data, and calculate the spectral centroid variance of each music sample data based on the spectral centroid sequence corresponding to each music sample data; calculate the mean of the spectral centroid variance of each music sample data in the music data sample set as the third mean; divide each noise sample data in the noise data sample set into multiple frames, determine the spectral centroid point of each frame to obtain the spectral centroid sequence corresponding to each noise sample data, and calculate the spectral centroid variance of each noise sample data based on the spectral centroid sequence corresponding to each noise sample data; calculate the mean of the spectral centroid variance of each noise sample data in the noise data sample set as the fourth mean; and take half the sum of the third mean and the fourth mean as the centroid variance threshold.

[0085] For example: Calculate the centroid variance of the spectrum of all music samples in the music data sample set and obtain the mean n1; calculate the centroid variance of the spectrum of all noise samples in the noise data sample set and obtain the mean n2. The mean of both is b = (n1 + n2) / 2. During the prediction phase, if the centroid variance of the spectrum of the music sample to be processed is greater than b, the score can be set to 1; if it is less than b, it can be set to -1.

[0086] S3: Calculate the third score of the representation features in the latent space;

[0087] For example, taking the embedding (embedding layer output) as the representation feature in the latent space as an example, calculating the third score of the representation feature in the latent space can include:

[0088] S3-1: Extract the Mel frequency cepstral coefficient features of the music sample to be processed;

[0089] S3-2: Input the Mel frequency cepstral coefficient features into a pre-established autoencoder network to obtain the embedding of the music sample to be processed;

[0090] S3-3: Calculate the distance between the embedding of the music sample to be processed and the mean of the music embedding and the mean of the noise embedding;

[0091] S3-4: When the embedding of the music sample to be processed is close to the mean of the music embedding, the first distance value is used as the third score; when the embedding of the music sample to be processed is close to the mean of the noise embedding, the second distance value is used as the third score. The first distance value represents a low noise probability, and the second distance value represents a high noise probability.

[0092] The aforementioned mean music embedding and mean noise embedding can be set as follows: Obtain a music data sample set and a noise data sample set; extract the Mel-frequency cepstral coefficient features of each music data sample in the music data sample set, input the Mel-frequency cepstral coefficient features of each music data sample into a pre-established autoencoder network to obtain the embedding of each music data sample, and calculate the mean of the embedding of each music data sample as the mean music embedding; extract the Mel-frequency cepstral coefficient features of each noise data sample in the noise data sample set, input the Mel-frequency cepstral coefficient features of each noise data sample into a pre-established autoencoder network to obtain the embedding of each noise data sample, and calculate the mean of the embedding of each noise data sample as the mean noise embedding.

[0093] For example, all music samples and noise samples from the music data sample set and noise data sample set are input into the network. The mean embedding of the music data sample (emb1) and the mean embedding of the noise data sample (emb2) are calculated respectively. These two embeddings represent their positions in the feature space. In the prediction stage, the MFCC features of the music sample to be processed are extracted and input into the network trained above to obtain the embedding. The Euclidean distance between the current embedding and emb1 and emb2 is calculated. If the distance to emb1 is closer, the score is set to 1; otherwise, the score is set to -1.

[0094] S4: According to the pre-set weight ratio, the first score, the second score, and the third score are weighted and summed to obtain the target score;

[0095] S5: Determine whether the target score is within a preset noise range. If the target score is within a preset noise range, then determine that the music sample to be processed is music noise.

[0096] For example, the weighted score of the three factors can be calculated using the following formula to obtain the final total score:

[0097] Score=k1*score1+k2*score2+k3*score3.

[0098] Here, score1 represents the first score, score2 represents the second score, and score3 represents the third score. k1, k2, and k3 represent the weighting coefficients of the three features, which control the proportion of each feature in the final score. If the score is greater than 0, it is judged as music; if the score is less than or equal to 0, it is judged as noise.

[0099] However, it is worth noting that the aforementioned representation features in the time domain, spectrum space, and latent space are merely exemplary descriptions. In implementation, other representation features can be selected as representation features in the time domain, spectrum space, and latent space, depending on the situation and requirements. For example, in addition to selecting the zero-crossing rate as the representation feature in the time domain, short-time energy, autocorrelation function, etc., can also be selected as representation features in the time domain. In addition to selecting the spectral centroid variance as the representation feature in the spectrum space, the mean of spectral components can also be selected as the representation feature in the spectrum space.

[0100] Furthermore, it is worth noting that the above-mentioned score values ​​and their settings are merely illustrative descriptions. In actual implementation, other values ​​can be used as score values. For example, the above-mentioned value of 1 can be set to -1, and the value of -1 can be set to 1. Correspondingly, the judgment method is also converted. The above-mentioned values ​​can also be set according to 10, -10, etc. This application does not limit the specific score value method, and it can be set according to needs and circumstances.

[0101] The music noise sample filtering method presented in this example can be used in scenarios such as cover song recognition, timbre recognition, multi-dimensional scoring, song recognition by listening, and humming recognition. Musical noise directly affects model performance. For example, if noise samples are incorporated into a cover song recognition model, the model will learn the mapping relationship from noise to song. However, the noise space is often random, which may affect the mapping relationship between songs, leading to a decrease in model recognition performance. Ultimately, this results in inaccurate recognition results and a poor user experience. Therefore, this example proposes an effective noise discrimination method using multi-feature fusion, ensuring that the training of various music models is not interfered with by noise, improving model performance, and effectively enhancing the user experience.

[0102] The above method will be described below with reference to a specific embodiment. However, it is worth noting that this specific embodiment is only for better illustration of this application and does not constitute an improper limitation of this application.

[0103] The general process for filtering music noise samples is as follows: First, collect the training samples needed for the binary classification model. These training samples include music samples and task-irrelevant non-music samples. For non-music samples, to ensure the comprehensiveness of the noise spatial distribution in the training samples, three main types of noise need to be collected: The first is scene noise, including airports, bus stops, homes, parks, shopping malls, etc. The more scenes, the more comprehensive the noise spatial representation; the second is human voice noise, including human voice noise unrelated to singing, noisy human voices, television voices, etc.; the third is instrument noise. Not all audio containing instruments belongs to music. Music has melody, and instrument noise is often easily confused with music; therefore, instrument noise should also be included in the non-music samples. After the sample collection is complete, spectral features can be extracted from the samples and input into the binary classification model for training. The model learns the differences in the feature spatial distribution of music samples and noise samples, and distinguishes between the two.

[0104] However, while this approach is task-oriented and has a clear objective, the performance of binary classification models depends heavily on training samples, especially non-musical samples. Noise spaces are diverse and complex; the training samples' ability to describe a given noise space limits the model's performance. Therefore, if the model cannot fully encompass all noise types encountered in real-world scenarios, it struggles to achieve satisfactory results. Furthermore, in music scenarios, the spectral spaces of certain musical information and noise overlap, making direct differentiation impossible for binary classification models. Therefore, considering the characteristics of music noise scenarios, this example employs feature fusion to distinguish between music noise and non-musical noise based on characteristics between them. This multi-feature fusion method for filtering music noise samples is proposed, not entirely reliant on the data.

[0105] This example proposes a music noise sample filtering method based on multi-feature fusion. For instance, it combines zero-crossing rate, spectral centroid sequence, and autoencoder embedding features for noise sample filtering. Specifically, a batch of music samples and non-music samples can be collected, and the mean zero-crossing rate of the music samples and non-music samples can be calculated separately. The mean of the two mean zero-crossing rates, 'a', is used as the scoring threshold. Then, the mean variance of the spectral centroid sequences of the music samples and non-music samples can be calculated separately, and the mean of the two mean variances, 'b', is used as the scoring threshold. Furthermore, an autoencoder network is trained to describe the spatial distribution of the embedding features of the two samples. After training, the music and non-music samples are input into the autoencoder network, and the mean embeddings of the two samples are extracted separately. The distance between a single sample and the two mean embeddings is calculated, and the distance is fused with the other two features in the form of a score. Finally, the three scores are weighted to determine whether the predicted sample belongs to music noise, and then filtered or retained accordingly.

[0106] To illustrate with a specific embodiment, the above-described music noise sample filtering method based on multi-feature fusion can be implemented as follows: Figure 2 As shown, it includes the following steps:

[0107] S1: Collect music samples and noise samples. The noise samples consist of various non-musical noises, including: scene noise, irrelevant human voices, instrument noise, etc.

[0108] S2: Extract the zero-crossing rate of the music samples and the zero-crossing rate of the noise samples respectively;

[0109] Zero-crossing rate represents the number of times each frame of the signal crosses a zero point, reflecting the randomness of the signal. Noise signals are more random and unpredictable than music signals, therefore their zero-crossing rate is higher. Assuming the average zero-crossing rate of all music samples is m1 and the average zero-crossing rate of all noise samples is m2, the zero-crossing rate threshold can be set as a = (m1 + m2) / 2. During the prediction phase, if the zero-crossing rate of the sample to be predicted is greater than a, the score can be set to -1; if the zero-crossing rate is less than a, the score can be set to 1.

[0110] S3: Extract the centroid variance of the spectrum of the music sample and the centroid variance of the spectrum of the noise sample respectively.

[0111] The centroid of the spectrum is a crucial physical parameter describing timbre attributes; it represents the centroid of frequency components. Each frame can have a centroid point extracted, indicating the dominant frequency component of that frame. Therefore, an audio clip can be divided into multiple frames, from which a centroid sequence can be extracted. The centroid sequence is highly effective at distinguishing noise from single-musical instruments because instrumental noise lacks melody, exhibits minimal local spectral fluctuations, and has a relatively stable centroid sequence. Furthermore, noise samples are generally concentrated in low frequencies, making their centroid sequences more stable than those of music. Therefore, the centroid variance of all music samples is calculated, and its mean n1 is obtained. Similarly, the centroid variance of all noise samples is calculated, and its mean n2 is obtained. The mean of both is b = (n1 + n2) / 2. During the prediction phase, if the centroid variance of the sample to be predicted is greater than b, a score of 1 can be set; otherwise, a score of -1 can be set.

[0112] In practice, the mean of the spectral components can be calculated instead of the variance of the spectral centroid.

[0113] S4: Training the autoencoder network, specifically:

[0114] Randomly select music-music sample pairs and noise-noise sample pairs from the training samples. Select one of the sample pairs, extract the spectral features Mel-frequency cepstral coefficients (MFCC) and input them into the network to obtain the embedding x' in the output layer. Calculate the loss function using the original MFCC features x of the other sample.

[0115]

[0116] The loss function is the Euclidean distance between the two samples, representing the mapping relationship in the spectral space. For both music samples and noise samples, a smaller distance is better. The network is trained until it converges to L, and the trained network is saved. In implementation, besides using Euclidean distance as the loss function, cosine distance can also be chosen.

[0117] The aforementioned autoencoder network can be a convolutional autoencoder network, a fully connected autoencoder network, or other spectral features can be used to replace the MFCC features.

[0118] Input all music samples and noise samples into the network, and calculate the mean embedding value emb1 of the music samples and the mean embedding value emb2 of the noise samples respectively. These two embeddings represent the positions of the two in the feature space.

[0119] During the prediction phase, the MFCC features of the prediction samples are extracted and input into the network trained above to obtain the embedding. The Euclidean distance between the current embedding and emb1 and emb2 is calculated. If the current embedding is closer to emb1, the score is set to 1; otherwise, the score is set to -1.

[0120] S5: Calculate the weighted score of the three scores according to the following formula to obtain the final total score:

[0121] Score=k1*score1+k2*score2+k3*score3.

[0122] Here, score1 represents the score in step S2, score2 represents the score in step S3, score3 represents the score in step S4, and k1, k2, and k3 represent the weighting coefficients of the three features, which can control the proportion of the three features in the final score.

[0123] If the score is greater than 0, it is judged as music; if the score is less than or equal to 0, it is judged as noise.

[0124] However, it is worth noting that the above-mentioned score values ​​and their settings are merely illustrative descriptions. In actual implementation, other values ​​can be used as score values. For example, the above-mentioned value of 1 can be set to -1, and the value of -1 can be set to 1. Correspondingly, the judgment method is also converted. The above-mentioned values ​​can also be set according to 10, -10, etc. This application does not limit the specific score value method and can set it according to needs and circumstances.

[0125] In the example above, a music noise sample filtering method based on multi-feature fusion was proposed. Starting from the characteristics of noise samples, a variety of effective features were adopted. It does not rely on complex and diverse noise samples to construct the feature space, so it is more efficient and faster to implement. Compared with binary classification noise filtering methods, this example distinguishes noise and music by using the zero-crossing rate and the variance of the spectral centroid, based on the differences in the characteristics of noise and music in the spectral space and temporal space. In order to describe the potential feature space distribution, an autoencoder network is used to learn the potential features, and the embedding center is used to distinguish between the two. Finally, the score adopts a weighted fusion method, which allows the proportion of the three features to be flexibly changed in different scenarios, making noise filtering more flexible.

[0126] This invention provides a method and apparatus for filtering music noise samples, as well as related electronic devices and storage media. The music noise sample filtering method can be implemented using one or more computers, such as terminals, like mobile terminals, or smartphones. In some embodiments, the music noise sample filtering apparatus can be implemented by software, hardware, or a combination of both.

[0127] In such Figure 3 In the exemplary embodiment shown, a music noise sample filtering device 300 is also provided. The music noise sample filtering device 300 may include:

[0128] The acquisition module 310 is used to acquire music samples to be processed;

[0129] Extraction module 320 is used to extract the representation features of the music sample to be processed in the time domain space, the representation features in the spectrum space, and the representation features in the latent space.

[0130] The determination module 330 is used to perform weighted fusion processing on the representation features in the time domain space, the representation features in the spectral space, and the representation features in the latent space to determine the target score;

[0131] The filtering module 340 is used to filter the music sample to be processed when the target score meets the preset condition, that is, when the music sample to be processed is music noise.

[0132] In one embodiment, the determining module 303 may specifically include: a first calculation unit for calculating a first score of the representation features in the time domain; a second calculation unit for calculating a second score of the representation features in the spectral space; a third calculation unit for calculating a third score of the representation features in the latent space; a fourth calculation unit for weighted summation of the first score, the second score, and the third score according to a preset weight ratio to obtain a target score; and a determining unit for determining whether the target score meets a preset condition. If the target score meets the preset condition, the music sample to be processed is determined to be music noise.

[0133] In one embodiment, the first computing unit is specifically used to extract the zero-crossing rate of the music sample to be processed; compare the zero-crossing rate with a preset zero-crossing threshold; if the zero-crossing rate is greater than the zero-crossing threshold, use a first zero-crossing value as the first score; if the zero-crossing rate is not greater than the zero-crossing threshold, use a second zero-crossing value as the first score, wherein the first zero-crossing value represents a high noise probability and the second zero-crossing value represents a low noise probability.

[0134] In one implementation, the zero-crossing threshold can be set as follows:

[0135] Obtain music data sample sets and noise data sample sets;

[0136] The mean zero-crossing rate of the data samples in the music data sample set is determined as the first mean;

[0137] The mean zero-crossing rate of the data samples in the noise data sample set is determined as the second mean;

[0138] The sum of the first mean and the second mean is taken as the zero-crossing threshold.

[0139] In one embodiment, the second computing unit can be specifically used to extract the spectral centroid variance of the music sample to be processed; compare the spectral centroid variance with a preset centroid threshold; if the spectral centroid variance is greater than the preset centroid variance threshold, use the first centroid value as the second score; if the spectral centroid variance is not greater than the centroid variance threshold, use the second centroid value as the second score, wherein the first centroid value represents a low noise probability and the second centroid value represents a high noise probability.

[0140] In one implementation, the centroid variance threshold can be set as follows:

[0141] Obtain music data sample sets and noise data sample sets;

[0142] The music data sample set is divided into multiple frames, and the spectral centroid point of each frame is determined to obtain the spectral centroid sequence corresponding to each music data sample. Based on the spectral centroid sequence corresponding to each music data sample, the spectral centroid variance of each music data sample is calculated.

[0143] The mean of the centroid variance of the spectrum of each music sample in the music data sample set is obtained and used as the third mean.

[0144] The noise data sample set is divided into multiple frames, and the spectral centroid point of each frame is determined to obtain the spectral centroid sequence corresponding to each noise data sample. Based on the spectral centroid sequence corresponding to each noise data sample, the spectral centroid variance of each noise data sample is calculated.

[0145] The mean of the centroid variance of the spectrum of each noise sample in the noise data sample set is obtained as the fourth mean.

[0146] The sum of the third mean and the fourth mean is taken as the centroid variance threshold.

[0147] In one embodiment, the third computing unit can specifically be used to extract the Mel frequency cepstral coefficient features of the music sample to be processed; input the Mel frequency cepstral coefficient features into a pre-established autoencoder network to obtain the embedding of the music sample to be processed; calculate the distance between the embedding of the music sample to be processed and the mean of the music embedding and the mean of the noise embedding; if the embedding of the music sample to be processed is close to the mean of the music embedding, a first distance value is used as the third score; if the embedding of the music sample to be processed is close to the mean of the noise embedding, a second distance value is used as the third score, wherein the first distance value represents a low noise probability and the second distance value represents a high noise probability.

[0148] In one implementation, the mean music embedding and the mean noise embedding can be set as follows:

[0149] Obtain music data sample sets and noise data sample sets;

[0150] Mel frequency cepstral coefficient features of each music data sample in the music data sample set are extracted. The Mel frequency cepstral coefficient features of each music data sample are input into a pre-established autoencoder network to obtain the embedding of each music data sample. The mean of the embedding of each music data sample is calculated as the music embedding mean.

[0151] The Mel frequency cepstral coefficient features of each noise data sample in the noise data sample set are extracted. The Mel frequency cepstral coefficient features of each noise data sample are input into a pre-established autoencoder network to obtain the embedding of each noise data sample. The mean of the embedding of each noise data sample is calculated as the noise embedding mean.

[0152] An embodiment of the present invention provides an electronic device comprising: a processor and a memory storing a computer program, the processor being configured to implement any method according to an embodiment of the present invention when running the computer program. Additionally, a processing apparatus for implementing an embodiment of the present invention may also be provided.

[0153] In a preferred embodiment of the present invention, the electronic device is a mobile terminal, preferably a mobile phone. This is merely an exemplary implementation. Figure 4 A schematic diagram of the hardware structure of a specific embodiment of an electronic device, such as a mobile terminal 800, is shown; while Figure 5 and Figure 6 A schematic diagram of the system architecture of a specific embodiment of an electronic device, such as a mobile terminal, is shown.

[0154] In the illustrated embodiment, the mobile terminal 800 may include a processor 801, an external memory interface 812, an internal memory 810, a universal serial bus (USB) interface 813, a charging management module 814, a power management module 815, a battery 816, a mobile communication module 840, a wireless communication module 842, antennas 839 and 841, an audio module 834, a speaker 835, a receiver 836, a microphone 837, a headphone jack 838, buttons 809, a motor 808, an indicator 807, a subscriber identity module (SIM) card interface 88, a display screen 805, a camera device 806, and a sensor module 820, etc.

[0155] It is understood that the structure illustrated in the embodiments of this application does not constitute a specific limitation on the mobile terminal 800. In other embodiments of this application, the mobile terminal 800 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0156] In some embodiments, processor 801 may include one or more processing units. In some embodiments, processor 801 may include one or a combination of at least two of the following: application processor (AP), modem processor, baseband processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, neural network processor (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0157] The controller can serve as the nerve center and command center of the mobile terminal 800. Based on the instruction opcode and timing signals, the controller can generate operation control signals to control the fetching and execution of instructions.

[0158] The processor may also include memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or that are used repeatedly. If the processor needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 801, and thus improves the efficiency of the system.

[0159] NPU stands for Neural Network (NN) Computing Processor. By drawing inspiration from the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can quickly process input information and continuously learn on its own.

[0160] A GPU is a microprocessor for image processing, connected to the display screen and application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. A processor may include one or more GPUs, which execute program instructions to generate or modify display information.

[0161] A digital signal processor (ISP) is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals.

[0162] In some embodiments, the processor 801 may include one or more interfaces. The interface may include an integrated circuit (I2C) interface, an integrated circuit built-in audio (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous transceiver (UART) interface, a mobile industry processor interface (MIPI), a general purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, a universal serial bus (USB) interface, etc.

[0163] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a limitation on the structure of the mobile terminal. In other embodiments of this application, the mobile terminal may also adopt different interface connection methods or a combination of multiple interface connection methods as described in the above embodiments.

[0164] The wireless communication function of the mobile terminal 800 can be implemented through antennas 839 and 841, mobile communication module 840, wireless communication module 842, modem processor or baseband processor, etc.

[0165] Video codecs are used to compress or decompress digital video.

[0166] Mobile terminal 800 can implement audio functions through audio modules, speakers, receivers, microphones, headphone jacks, and application processors. Examples include music playback and recording.

[0167] The audio module is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals.

[0168] A microphone is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can put their mouth close to the microphone to speak, inputting the sound signal into the microphone.

[0169] Sensor module 820 may include one or more of the following sensors:

[0170] The pressure sensor 823 is configured to sense pressure signals and convert them into electrical signals.

[0171] The 824 bar pressure sensor is used to measure bar pressure.

[0172] The magnetic sensor 825 includes a Hall sensor.

[0173] The gyroscope sensor 827 can be used to determine the motion posture of the mobile terminal 800.

[0174] The accelerometer 828 can detect the magnitude of the acceleration of the mobile terminal 800 in various directions.

[0175] The distance sensor 829 can be configured to measure distance.

[0176] The proximity light sensor 821 may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode.

[0177] The ambient light sensor 822 is used to sense ambient light intensity.

[0178] The fingerprint sensor 831 can be configured to collect fingerprints.

[0179] The touch sensor 832 can be located on the display screen, and the touch sensor and the display screen together form a touch screen, also known as a "touchscreen". The touch sensor is used to detect touch operations applied to or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event, such as single click, double click, long press, tap, directional swipe, pinch, etc.

[0180] The bone conduction sensor 833 can acquire vibration signals.

[0181] Software operating systems for electronic devices (computers), such as mobile terminals, can adopt layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture.

[0182] The embodiments shown in this article use the iOS and Android operating system platforms as examples of a layered architecture to illustrate the software structure of a mobile terminal. However, it is conceivable that the embodiments described herein can be implemented in different software operating systems.

[0183] exist Figure 5 In the illustrated embodiment, the solution of this invention can employ the iOS operating system. The iOS operating system adopts a four-layer architecture, from top to bottom: the Cocoa Touch layer 910, the Media layer 920, the Core Services layer 930, and the Core OS layer 940. The Touch layer 910 provides various commonly used frameworks for application development, and most of these frameworks are related to the user interface, handling user touch interactions on the iOS device. The Media layer provides audiovisual technologies for applications, such as graphics, sound technology, video, and frameworks related to audio and video transmission. The Core Services layer provides the basic system services required by applications. The Core OS layer contains most low-level, near-hardware functionalities.

[0184] In this embodiment of the invention, UIKit is the user interface framework of the touchable layer 910, which can be supported by numerous image frames in the media layer 920, including but not limited to... Figure 5 The components shown are Core Graphics, Core Animation, Open GL ES, Core Image, ImageIO, and GLKit.

[0185] Figure 6This is a schematic diagram of the Android operating system architecture. The solution of this embodiment of the invention can use the Android operating system. The layered architecture divides the software into several layers, which communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: application layer 1010, application framework layer 1020, Android runtime and system library 1030, and kernel layer 1040.

[0186] Application layer 1010 may include a series of application packages.

[0187] The Application Framework Layer 1020 provides Application Programming Interfaces (APIs) and a programming framework for applications within the Application Layer. The Application Framework Layer includes several predefined functions.

[0188] The window manager is used to manage window programs.

[0189] Content providers are used to store and retrieve data, and make that data accessible to applications.

[0190] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0191] The phone manager is used to provide communication functions for mobile terminals.

[0192] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0193] The notification manager allows applications to display notification information in the status bar. It can be used to convey informational messages and can disappear automatically after a short time without user interaction.

[0194] The Android Runtime comprises the core libraries and the virtual machine, and is responsible for the scheduling and management of the Android system. The core libraries consist of two parts: one part contains the functionalities that Java calls, and the other part contains the core Android libraries. The application layer and framework layer run within the virtual machine.

[0195] The system library can include multiple functional modules. The surface manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0196] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0197] Kernel layer 1040 is the layer between hardware and software. The kernel layer may include display drivers, camera drivers, audio interfaces, sensor drivers, power management, and GPS interfaces. In some embodiments of the invention, the display can invoke the display driver.

[0198] The systems, devices, modules, or units described in the above embodiments can be implemented by electronic devices (computers) or their associated components, preferably by mobile terminals. Mobile terminals can be, for example, smartphones, laptops, in-vehicle human-machine interaction devices, personal digital assistants, media players, navigation devices, game consoles, tablets, wearable devices, or combinations thereof.

[0199] Although not shown, some embodiments also provide a storage medium storing a computer program. The computer program is configured to be executed, when run, to perform the methods of any embodiment of the invention.

[0200] Storage media in embodiments of the present invention include articles that are permanent and non-permanent, removable and non-removable, capable of storing information by any method or technology. Examples of storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0201] Those skilled in the art will understand that the embodiments described in this specification can be provided as methods, systems, or computer program products. Therefore, those skilled in the art will realize that the functional modules / units or controllers and related method steps described in the above embodiments can be implemented in software, hardware, or a combination of both.

[0202] Unless explicitly stated otherwise, the actions or steps of the methods and procedures described in the embodiments of the present invention do not necessarily have to be performed in a specific order and can still achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0203] This document describes several embodiments of the present invention; however, for the sake of brevity, the descriptions of the embodiments are not exhaustive, and identical or similar features or parts between the embodiments may be omitted. In this document, "one embodiment," "some embodiments," "example," "specific example," or "some examples" refers to embodiments applicable to at least one, but not all, of the present invention. The above terms do not necessarily refer to the same embodiments or examples. Without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described herein, as well as the features of the different embodiments or examples.

[0204] The exemplary systems and methods of the present invention have been specifically shown and described with reference to the foregoing embodiments, which are merely examples of the best mode for implementing the systems and methods. Those skilled in the art will understand that various changes can be made to the embodiments of the systems and methods described herein without departing from the spirit and scope of the invention as defined in the appended claims when implementing the systems and / or methods.

Claims

1. A method for filtering music noise samples, characterized in that, include: Obtain the music sample to be processed; Extract the representation features of the music sample to be processed in the time domain, the representation features in the spectral space, and the representation features in the latent space, wherein the latent space is a low-dimensional feature space; The target score is determined by weighted fusion of the representation features in the time domain, the representation features in the spectral space, and the representation features in the latent space. If the target score meets the preset conditions, then the music sample to be processed is music noise, and the music sample to be processed is filtered. Specifically, the target score is determined by weighted fusion of the representation features in the time domain, the representation features in the spectral space, and the representation features in the latent space, including: Calculate a first score for the representation features in the time domain space, wherein the representation features in the time domain space include the zero-crossing rate; Calculate a second score for the characterization features in the spectral space, wherein the characterization features in the spectral space include the spectral centroid variance; Calculate the third score of the representation features in the latent space, the representation features of the latent space including the embedding layer output; Based on the pre-set weight ratios, the first score, the second score, and the third score are weighted and summed to obtain the target score. If the target score is determined to meet the preset conditions, then the music sample to be processed is determined to be music noise.

2. The method according to claim 1, characterized in that, When the representation feature in the time domain is a zero-crossing rate, the first score of the representation feature in the time domain is calculated, including: Extract the zero-crossing rate of the music sample to be processed; The zero-crossing rate is compared with a preset zero-crossing threshold; When the zero-crossing rate is greater than the zero-crossing threshold, the first zero-crossing value is used as the first score; when the zero-crossing rate is not greater than the zero-crossing threshold, the second zero-crossing value is used as the first score. The first zero-crossing value represents a high noise probability, and the second zero-crossing value represents a low noise probability.

3. The method according to claim 2, characterized in that, Set the zero-crossing threshold as follows: Obtain music data sample sets and noise data sample sets; The mean zero-crossing rate of the data samples in the music data sample set is determined as the first mean; The mean zero-crossing rate of the data samples in the noise data sample set is determined as the second mean; The sum of the first mean and the second mean is taken as the zero-crossing threshold.

4. The method according to claim 1, characterized in that, When the representation feature in the spectral space is the spectral centroid variance, the second score of the representation feature in the spectral space is calculated, including: Extract the spectral centroid variance of the music sample to be processed; The spectral centroid variance is compared with a preset centroid variance threshold. If the spectral centroid variance is greater than the centroid variance threshold, the first centroid value is used as the second score; if the spectral centroid variance is not greater than the centroid variance threshold, the second centroid value is used as the second score. The first centroid value represents a low noise probability, and the second centroid value represents a high noise probability.

5. The method according to claim 4, characterized in that, Set the centroid variance threshold as follows: Obtain music data sample sets and noise data sample sets; The music data sample set is divided into multiple frames, and the spectral centroid point of each frame is determined to obtain the spectral centroid sequence corresponding to each music data sample. Based on the spectral centroid sequence corresponding to each music data sample, the spectral centroid variance of each music data sample is calculated. The mean of the centroid variance of the spectrum of each music sample in the music data sample set is obtained and used as the third mean. The noise data sample set is divided into multiple frames, and the spectral centroid point of each frame is determined to obtain the spectral centroid sequence corresponding to each noise data sample. Based on the spectral centroid sequence corresponding to each noise data sample, the spectral centroid variance of each noise data sample is calculated. The mean of the centroid variance of the spectrum of each noise sample in the noise data sample set is obtained as the fourth mean. The sum of the third mean and the fourth mean is taken as the centroid variance threshold.

6. The method according to claim 1, characterized in that, When the representation features in the latent space are the outputs of the embedding layer, the third score of the representation features in the latent space is calculated, including: Extract the Mel frequency cepstral coefficient features of the music sample to be processed; The Mel frequency cepstral coefficient features are input into a pre-established autoencoder network to obtain the embedding layer output of the music sample to be processed. Calculate the distance between the embedding layer output of the music sample to be processed and the mean output of the music embedding layer and the mean output of the noise embedding layer; When the distance between the embedding layer output of the music sample to be processed and the average output of the music embedding layer is close, the first distance value is used as the third score. When the distance between the embedding layer output of the music sample to be processed and the average output of the noise embedding layer is close, the second distance value is used as the third score. The first distance value represents a low noise probability, and the second distance value represents a high noise probability.

7. The method according to claim 6, characterized in that, Set the output mean of the music embedding layer and the output mean of the noise embedding layer as follows: Obtain music data sample sets and noise data sample sets; Mel frequency cepstral coefficient features of each music data sample in the music data sample set are extracted. The Mel frequency cepstral coefficient features of each music data sample are input into a pre-established autoencoder network to obtain the embedding layer output of each music data sample. The mean of the embedding layer output of each music data sample is calculated as the mean of the music embedding layer output. The Mel frequency cepstral coefficient features of each noise data sample in the noise data sample set are extracted. The Mel frequency cepstral coefficient features of each noise data sample are input into a pre-established autoencoder network to obtain the embedding layer output of each noise data sample. The mean of the embedding layer output of each noise data sample is calculated as the mean of the noise embedding layer output.

8. An electronic device, characterized in that, include: A processor and a memory storing a computer program, the processor being configured to perform the method of any one of claims 1 to 7 when running the computer program.

9. A storage medium, characterized in that, The storage medium stores a computer program configured to execute the method of any one of claims 1 to 7 when run.

Images