Music genre identification method, electronic equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a music genre identification method, electronic equipment and a storage medium. According to the music genre identification method, a to-be-identified target sound source file needs to be acquired firstly, then audio sampling processing is carried out on the target sound source file to obtain a target Mel frequency spectrum, sound source separation processing is further carried out on the target Mel frequency spectrum to obtain a target sub-frequency spectrum, and the target sub-frequency spectrum is identified according to the target sub-frequency spectrum. The target sub-spectrum is used for representing one type of sound in the target sound source file, and further, inputting the target sub-spectrum into a pre-trained audio processing model for genre identification processing to obtain a music genre category corresponding to the target audio file. Because the target Mel spectrum needs to be subjected to sound source separation processing before being input into the audio processing model to obtain the target sub-spectrum, music genre classification is carried out based on each type of sound, and the accuracy of music genre identification can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a music genre recognition method, electronic device, and storage medium. Background Art

[0002] Music Genre Classification refers to the task of identifying the music genre to which a given music track belongs. Its input is the original audio file, and its output is the music genre classification. Currently, the commonly used music service providers in the industry all have the function of using music genres to classify, retrieve, or recommend music.

[0003] In the related art, most music genre recognition methods still rely on manually annotated labels, and some music genre recognition methods based on deep learning or traditional machine learning have too low recognition accuracy when dealing with music with more complex arrangements. Therefore, how to improve the accuracy of music genre recognition has become a problem that needs to be solved urgently in the industry. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a music genre recognition method, an electronic device, and a storage medium, which can improve the accuracy of music genre recognition.

[0005] According to the first aspect of the present application, the music genre identification method includes:

[0006] Obtain the target audio source file to be identified;

[0007] Performing audio sampling processing on the target sound source file to obtain a target Mel spectrum;

[0008] Performing sound source separation processing on the target Mel spectrum to obtain a target sub-spectrum, where the target sub-spectrum is used to characterize a type of sound in the target sound source file;

[0009] The target sub-spectrum is input into a pre-trained audio processing model for genre recognition processing to obtain the music genre category corresponding to the target audio file.

[0010] According to some embodiments of the present application, performing sound source separation processing on the target Mel spectrum to obtain a target sub-spectrum includes:

[0011] The target Mel spectrum is input into a pre-trained sound source separation model for sound source separation processing to obtain a plurality of target sub-spectra.

[0012] According to some embodiments of the present application, the audio processing model includes a feature extraction module, a sound track feature splicing module and a fully connected layer classifier, and the target sub-spectrum is input into a pre-trained audio processing model for genre recognition processing to obtain the music genre category corresponding to the target audio file, including:

[0013] Performing feature extraction on each of the target sub-spectrums based on the feature extraction module to obtain a plurality of audio feature sub-vectors;

[0014] Based on the sound track feature splicing module, a plurality of the audio feature sub-vectors are spliced ​​to obtain an audio feature total vector;

[0015] Based on the fully connected layer classifier, genre classification processing is performed on the total audio feature vector to obtain a music genre category corresponding to the target audio file.

[0016] According to some embodiments of the present application, performing audio sampling processing on the target sound source file to obtain a target Mel spectrum includes:

[0017] Obtaining a preset spectrum sampling rate, a preset Fourier transform window length, a preset window step size, and a preset number of Mel filters;

[0018] Based on the spectrum sampling rate, the Fourier transform window length, the window step size and the number of Mel filters, a Mel feature extraction function is called to perform audio sampling on the target sound source file to obtain the target Mel spectrum.

[0019] According to some embodiments of the present application, before inputting the target sub-spectrum into a pre-trained audio processing model for genre recognition processing to obtain the music genre category corresponding to the target audio file, the method further includes pre-training the audio processing model, specifically including:

[0020] Acquire a training data set, wherein the training data set includes a plurality of audio samples, each of the audio samples is configured with a corresponding music genre label;

[0021] Performing audio sampling processing on the audio sample to obtain a training Mel spectrum;

[0022] Performing sound source separation processing on the training Mel spectrum to obtain a training sub-spectrum, wherein the training sub-spectrum is used to characterize a type of sound in the audio sample;

[0023] Iteratively training the audio processing model based on the training sub-spectrum and the music genre label;

[0024] When the audio processing model meets the first preset condition in iterative training, the trained audio processing model is obtained.

[0025] According to some embodiments of the present application, the iterative training of the audio processing model based on the training sub-spectrum and the music genre label includes:

[0026] In each round of the iterative training, the training sub-spectrum is classified and identified based on the audio processing model to obtain training classification data;

[0027] Comparing the difference between the training classification data and the music genre label to obtain classification deviation data;

[0028] After each round of iterative training, the model parameters of the audio processing model are updated based on the classification deviation data.

[0029] According to some embodiments of the present application, the classification deviation data is a loss function gradient value, and updating the model parameters of the audio processing model based on the classification deviation data includes:

[0030] When the loss function gradient value meets the second preset condition, the weight and bias of the audio processing model are adjusted based on the loss function gradient value to update the audio processing model.

[0031] According to some embodiments of the present application, the classification deviation data is a loss function output value, and when the audio processing model meets the first preset condition in iterative training, the trained audio processing model is obtained, including:

[0032] When the output value of the loss function is less than a preset threshold, it is determined that the audio processing model meets the first preset condition in iterative training, and the trained audio processing model is obtained.

[0033] In a second aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the music genre identification method as described in any one of the embodiments of the first aspect of the present application.

[0034] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement a music genre identification method as described in any one of the embodiments of the first aspect of the present application.

[0035] The music genre identification method, electronic device, and storage medium according to the embodiments of the present application have at least the following beneficial effects:

[0036] The music genre identification method of the present application needs to first obtain the target sound source file to be identified, then perform audio sampling processing on the target sound source file to obtain a target Mel spectrum, further perform sound source separation processing on the target Mel spectrum to obtain a target sub-spectrum, and the target sub-spectrum is used to characterize a type of sound in the target sound source file, and further, the target sub-spectrum is input into a pre-trained audio processing model for genre identification processing to obtain the music genre category corresponding to the target audio file. Since the target Mel spectrum needs to be subjected to sound source separation processing to obtain the target sub-spectrum before being input into the audio processing model, the accuracy of music genre identification can be improved by classifying the music genres based on the sounds of each category.

[0037] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0039] Figure 1 A schematic diagram of a flow chart of a music genre identification method provided in an embodiment of the present application;

[0040] Figure 2 for Figure 1 Schematic diagram of the process of step S102;

[0041] Figure 3 Another schematic diagram of a process flow of a music genre identification method provided in an embodiment of the present application;

[0042] Figure 4 for Figure 3 Schematic diagram of the process of step S304;

[0043] Figure 5 for Figure 4 Schematic diagram of the process of step S403;

[0044] Figure 6 for Figure 3 Schematic diagram of the process of step S305;

[0045] Figure 7 for Figure 1 Schematic diagram of the process of step S104;

[0046] Figure 8 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0047] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0048] In the description of this application, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0049] In the description of the present application, it should be understood that descriptions involving orientation, such as up, down, left, right, front, back, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0050] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0051] In the description of this application, it should be noted that, unless otherwise clearly defined, the terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in this application in combination with the specific content of the technical solution. In addition, the identification of specific steps below does not represent a limitation on the order of steps and execution logic. The execution order and execution logic between each step should be understood and inferred with reference to the contents described in the embodiment.

[0052] Music Genre Classification refers to the task of identifying the music genre to which a given music track belongs. Its input is the original audio file, and its output is the music genre classification. Currently, the commonly used music service providers in the industry all have the function of using music genres to classify, retrieve, or recommend music.

[0053] In terms of data management, music information systems based on music genres in the industry still mostly rely on manually annotated music genre tags, that is, staff annotate genre information for all songs in the database. This method is very time-consuming and cannot completely annotate all tracks.

[0054] Traditional automatic music genre recognition methods mainly use traditional machine learning methods on manually defined middle-level features, such as Mel-scale Frequency Cepstral Coefficients (MFCC), pitch, zero-crossing rate, etc. Traditional machine learning methods include support vector machines, decision trees, etc. It should be pointed out that the advantages of these methods are good generalization and less computing resources required, but the disadvantages are low recognition accuracy, poor performance in complex scenarios, and poor performance when there are many genres.

[0055] Some of the automatic music genre recognition methods currently used in the industry are also based on deep learning. Since the spectrum has many similarities with ordinary images, this type of method first extracts the Short-Time Fourier Transform (STFT) spectrum or Mel spectrum from the original music audio, and then uses common image processing models such as convolutional neural networks for feature extraction. Its advantage is that the accuracy is better than traditional methods, but its disadvantage is that the recognition performance of songs with more tracks and more complex arrangements is still poor, and a large amount of training data is required.

[0056] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a music genre recognition method, an electronic device, and a storage medium, which can improve the accuracy of music genre recognition, especially when identifying songs with more complex arrangements, and can show better performance.

[0057] Reference Figure 1 According to some optional embodiments of the music genre identification method shown in the embodiments of the present application, it may include but is not limited to the following steps S101 to S104.

[0058] Step S101, obtaining a target audio source file to be identified;

[0059] Step S102, performing audio sampling processing on the target sound source file to obtain a target Mel spectrum;

[0060] Step S103, performing sound source separation processing on the target Mel spectrum to obtain a target sub-spectrum, where the target sub-spectrum is used to represent a type of sound in the target sound source file;

[0061] Step S104: input the target sub-spectrum into a pre-trained audio processing model for genre recognition processing to obtain the music genre category corresponding to the target audio file.

[0062] In the embodiment of the present application shown by steps S101 to S104, it is necessary to first obtain the target sound source file to be identified, then perform audio sampling processing on the target sound source file to obtain a target Mel spectrum, further perform sound source separation processing on the target Mel spectrum to obtain a target sub-spectrum, and the target sub-spectrum is used to characterize a type of sound in the target sound source file, and further, the target sub-spectrum is input into a pre-trained audio processing model for genre recognition processing to obtain the music genre category corresponding to the target audio file. Since the target Mel spectrum needs to be subjected to sound source separation processing to obtain the target sub-spectrum before being input into the audio processing model, the accuracy of music genre recognition can be improved by classifying the music genres based on the sounds of each category.

[0063] In step S101 of some embodiments of the present application, a target sound source file to be identified is obtained. It should be noted that the target sound source file to be identified refers to the sound source file of the corresponding music genre to be identified. It should be understood that the music work is mainly generated by the superposition of various types of tracks in a specific time interval after arrangement. Specifically, the tracks of types such as vocals, bass, drums, guitars, etc. are pre-arranged and superimposed in a specific time interval, so as to achieve the changes and combinations of various aspects such as rhythm, rhythm, and melody during the playback process of the audio, and a music work is generated accordingly, and the sound source file in the embodiment of the present application refers to the audio file used to carry the music work. It should be pointed out that just because the music work is generated by the superposition of various types of tracks in a specific time interval after arrangement, the music work will have the difference between the complexity and simplicity of the arrangement. And the category characteristics of each music genre are significantly related to the superposition characteristics of each track in the music work. For example, in R&B music, the vocal part is often more prominent, the drum beats are simple and bright, and the bass part has a simple and repeated melody, often emphasizing the two-four beat. In some embodiments of the present solution, these tracks are separated and then analyzed to achieve more efficient music genre identification.

[0064] In step S102 of some embodiments of the present application, the target audio source file is subjected to audio sampling processing to obtain a target Mel spectrum. It should be noted that the target Mel spectrum refers to the Mel spectrum converted based on the target audio source file. Mel spectrum refers to the spectrum obtained after processing the audio file according to the Mel Scale. Human perception of frequency is not linear, and the perception of low-frequency signals is more sensitive than high-frequency signals. For example, people can easily find the difference between 500Hz and 1000Hz, but it is difficult to find the difference between 7500Hz and 8000Hz. At this time, the Mel Scale was proposed. It is a nonlinear transformation of Hz. For signals in Mel-Scale units, people can achieve almost the same perception ability for signals with the same frequency difference. Among them, the core of the Mel spectrum is the Mel scale, that is, the conversion standard for linear changes in frequency perception based on a logarithmic scale. The Mel spectrum contains three major characteristics, namely, the ability to tell relevant information in the time domain-frequency domain, the ability to perceive relevant amplitude information, and the ability to perceive relevant frequency domain information. It should be pointed out that, because the higher the frequency, the harder it is for the human ear to hear the difference when the sound frequency increases linearly, in some embodiments of the present application, the audio is described by a logarithmic spectrum such as a Mel spectrum instead of a linear spectrum. It should be noted that the librosa library is a third-party library for speech and signal processing in Python. In some exemplary embodiments of the present application, after the target sound source file is subjected to audio sampling processing, the librosa.feature.melspectrogram function in the librosa library is selected to process the sampled target audio, thereby further calculating the target Mel spectrum based on the target audio.

[0065] Reference Figure 2 According to the music genre identification method shown in some embodiments of the present application, step S102 may include but is not limited to the following steps S201 to S202.

[0066] Step S201, obtaining a preset spectrum sampling rate, a preset Fourier transform window length, a preset window step length, and a preset number of Mel filters;

[0067] Step S202: Based on the spectrum sampling rate, Fourier transform window length, window step size and the number of Mel filters, a Mel feature extraction function is called to perform audio sampling on the target sound source file to obtain a target Mel spectrum.

[0068] In steps S201 to S202 of some embodiments of the present application, it is necessary to first obtain a preset spectrum sampling rate, a preset Fourier transform window length, a preset window step size, and a preset number of Mel filters, and then based on the spectrum sampling rate, the Fourier transform window length, the window step size, and the number of Mel filters, call the Mel feature extraction function to perform audio sampling on the target sound source file to obtain the target Mel spectrum. It should be noted that in the process of performing audio sampling processing on the target sound source file, it is necessary to pre-set various types of sampling parameters, such as spectrum sampling rate, Fourier transform window length, window step size, and Mel filter number. During audio sampling processing, first obtain various types of sampling parameters, and then call the Mel feature extraction function based on the sampling parameters to perform audio sampling on the target sound source file to obtain the target Mel spectrum. It should be pointed out that there are various ways to obtain sampling parameters such as spectrum sampling rate, Fourier transform window length, window step size, and Mel filter number. They can be pre-stored in a database and the data of the sampling parameters can be called when they are used; the sampling parameters can also be obtained from other devices through network communication; the sampling parameters can also be obtained through input devices such as keyboards, mice, and touch screens. It should be understood that there are various ways to obtain sampling parameters such as spectrum sampling rate, Fourier transform window length, window step size and number of Mel filters, which may include, but not be limited to, the specific embodiments listed above. It should be clarified that the Mel feature extraction function refers to a function used to extract Mel features of a sound source file, and these Mel features may include but are not limited to relevant information of the time domain-frequency domain, audio amplitude information and audio frequency domain information in the sound source file.

[0069] In the embodiment of the present application shown in step S201 to step S202, the Mel feature extraction function is called to perform audio sampling on the target sound source file according to the pre-set sampling parameters, and a specific implementation method of sampling the target sound source file to form the corresponding target Mel spectrum is shown, which provides a reference basis for the implementation of the present method.

[0070] In some more specific embodiments. For the acquired target sound source file, firstly, a preset spectrum sampling rate, a preset Fourier transform window length, a preset window step size and a preset number of Mel filters are acquired, for example, the spectrum sampling rate is 16kHz, the Fourier transform window length is 2048, the window step size is 512, and the number of Mel filters is 80. Further, based on the spectrum sampling rate, the Fourier transform window length, the window step size and the number of Mel filters, the Mel feature extraction function is called to perform audio sampling on the target sound source file, for example, the librosa.feature.melspectrogram function in the librosa library is used to calculate the Mel spectrum corresponding to the target sound source file, and finally the target Mel spectrum is obtained.

[0071] In step S103 of some embodiments of the present application, the target Mel spectrum is subjected to sound source separation processing to obtain a target sub-spectrum, which is used to characterize a type of sound in the target sound source file. It should be noted that the iterative evolution of audio playback technology from mono, stereo, surround sound to three-dimensional sound is to restore the sound of the real world. Among them, three-dimensional sound technology uses a signal processing method to simulate the sound signals reaching both ears, and restores the sound field to a three-dimensional space, which is closer to the real world. At present, most of the audio that can be accessed in daily life is stereo, and various types of audio tracks (such as vocals, piano, guitar, etc. in music) have been mixed in the left and right channels and cannot be easily separated. In some embodiments of the present application, sound source separation can be achieved in a variety of ways, that is, various types of audio tracks are separated from a complete sound source file.

[0072] In some exemplary embodiments of the present application, the target Mel spectrum can be input into a pre-trained sound source separation model for sound source separation processing to obtain multiple target sub-spectra, wherein the sound source separation model can be a Demucs model based on the U-Net convolutional architecture and the Long Short-Term Memory (LSTM) network. It should be pointed out that in the application scenario of Demucs as a sound source separation model, Demucs and Convs-tasnet are provided as the backbone network implementation, which can be pre-trained on MusDB or other data sets. The pre-trained Demucs sound source separation model can separate drums, bass, vocals and other types of audio tracks to form multiple target sub-spectra, which is a more preferred solution compared to waveform or spectrogram-based methods.

[0073] In other embodiments of the present application, deep learning modeling can also be performed based on a large number of audio training files, and combined with traditional signal processing capabilities to ultimately achieve sound source separation: first, use Short-Time Fourier Transform (STFT) to transform the one-dimensional audio signal into a two-dimensional time-frequency spectrum, such as a Mel spectrum; after the one-dimensional audio signal is transformed into a two-dimensional time-frequency spectrum, the obtained two-dimensional time-frequency spectrum and the original one-dimensional time domain signal are used as dual-stream inputs, and through multi-layer residual coding and a large amount of data training, the latent space expression corresponding to the target audio track is obtained; finally, through a series of matrix transformations, sound source separation is achieved to obtain multiple target sub-spectra. This type of method includes three core steps: first, audio signal feature extraction: including directly extracting features from time domain signals through an encoder, and extracting time-frequency spectrum features from time domain signals through short-time Fourier transform; second, deep learning model construction: adding residual modules and attention mechanisms to enhance the ability to harmonically model different sound sources and the ability to correlate time series; third, multi-channel Wiener filtering: combining the capabilities of traditional signal processing, predicting the relationship between object and non-object power spectra through deep learning modeling, and constructing and processing filter coefficients.

[0074] It should be understood that there are various implementation schemes for performing sound source separation processing on the target Mel spectrum to obtain the target sub-spectrum, which may include but are not limited to the specific embodiments listed above.

[0075] According to the music genre identification method of some embodiments of the present application, the target Mel spectrum is subjected to sound source separation processing to obtain some implementation schemes of the target sub-spectrum. The target Mel spectrum can be input into a pre-trained sound source separation model for sound source separation processing to obtain multiple target sub-spectra. It should be noted that the target Mel spectrum refers to the Mel spectrum converted based on the target sound source file, and the target sound source file refers to the sound source file of the corresponding music genre to be identified. It should be emphasized that the audio that can be accessed in daily life is mostly stereo, and various types of tracks (such as vocals, piano, guitar, etc. in music) are already mixed in the left and right channels and cannot be easily separated. In order to separate the tracks corresponding to various types of sounds in the target Mel spectrum, some exemplary embodiments choose to input the target Mel spectrum into a pre-trained sound source separation model for sound source separation processing. It should be clarified that the sound source separation model refers to a neural network model with sound source separation capability, wherein the sound source separation capability possessed by the neural network model is obtained through pre-training. For example, the Demucs model based on the U-Net convolutional architecture and the Long Short-Term Memory (LSTM) network can separate the mel sub-spectra corresponding to the human voice, bass, drums and other parts from the mel spectrum after pre-training; for another example, the audio editing tool (Audio Editor Kit) based on HMS Core can obtain the one-dimensional audio signal after transforming it into a two-dimensional mel spectrum, and then use the obtained two-dimensional mel spectrum and the original one-dimensional time domain audio signal as dual-stream input, and obtain the latent space expression corresponding to the target audio track through multi-layer residual coding and training with a large amount of data; finally, through a series of matrix transformations, the sound source separation is realized to obtain multiple target sub-spectra. It should be understood that there are many types of sound source separation models, which can include, but are not limited to the specific embodiments listed above.

[0076] Reference Figure 3 According to the music genre identification method of some embodiments of the present application, before step S104, the music genre identification method of the embodiment of the present application also includes a pre-trained audio processing model, specifically including but not limited to the following steps S301 to S305.

[0077] Step S301, obtaining a training data set, the training data set including a plurality of audio samples, each audio sample being configured with a corresponding music genre label;

[0078] Step S302, performing audio sampling processing on the audio sample to obtain a training Mel spectrum;

[0079] Step S303, performing sound source separation processing on the training Mel spectrum to obtain a training sub-spectrum, where the training sub-spectrum is used to represent a type of sound in the audio sample;

[0080] Step S304, iteratively training the audio processing model based on the training sub-spectrum and the music genre label;

[0081] Step S305: When the audio processing model meets the first preset condition in iterative training, a trained audio processing model is obtained.

[0082] In step S301 of some embodiments of the present application, a training data set is obtained, and the training data set includes multiple audio samples, and each audio sample is configured with a corresponding music genre label. It should be noted that before step S104 of the embodiment of the present application, it is necessary to pre-train the audio processing model, and its purpose is to improve the ability of the audio processing model to recognize music genres. Therefore, it is necessary to use the training data set to train the ability of the audio processing model to recognize music genres. It should be pointed out that the types of audio samples in the training data set correspond to those of the target sound source files. If the target sound source file in the embodiment of the present application may be music of the R&B genre, then the audio samples in the training data set also need to include music of the R&B genre. Similarly, if the target sound source file in the embodiment of the present application may be music of the Jazz genre, then the audio samples in the training data set also need to include music of the Jazz genre. Examples are not given one by one here.

[0083] In some more specific embodiments, the training data set can be formed by collecting sound source files of various music genres and labeling them accordingly. A series of more typical data sets with music genre annotations, such as MusicDB, Million Song Dataset, GTZAN, etc., can also be used as the training data set of the embodiments of the present application.

[0084] In step S302 of some embodiments of the present application, audio sampling processing is performed on the audio sample to obtain a training Mel spectrum. It should be noted that since the type of data processed by the audio processing model in the embodiment of the present application in actual application is a Mel spectrum, the audio processing model also needs to perform audio sampling processing on the audio sample during pre-training to obtain a training Mel spectrum corresponding to the audio sample. Among them, the method of sampling the audio sample can also be to first obtain a preset sampling parameter, and then call the Mel feature extraction function to perform audio sampling on the audio sample based on the preset sampling parameters. In some more specific embodiments, the sampling parameters used when sampling the audio sample can be consistent with the sampling parameters used when sampling the target sound source file. It should be understood that since there are many ways to obtain a training Mel spectrum by performing audio sampling processing on the audio sample, the embodiment of the present application is not limited to extracting the training Mel spectrum through the specific embodiments cited above.

[0085] In step S303 of some embodiments of the present application, the training Mel spectrum is subjected to sound source separation processing to obtain a training sub-spectrum, and the training sub-spectrum is used to characterize a type of sound in the audio sample. It should be noted that, since the data processed by the audio processing model in the embodiment of the present application in actual application is the target sub-spectrum corresponding to each track after the sound source separation processing, the audio processing model also needs to perform sound source separation processing on the training Mel spectrum during pre-training to obtain training sub-spectrums corresponding to various types of sounds. Among them, the method of performing sound source separation processing on the training Mel spectrum can also be to input the training Mel spectrum into the pre-trained sound source separation model for sound source separation processing to obtain multiple training sub-spectra. In some more specific embodiments, the sampling parameters used when performing sound source separation processing on the training Mel spectrum can be consistent with the sound source separation model used when performing sound source separation processing on the target Mel spectrum, and the pre-training process that the sound source separation model has undergone in advance can also be consistent. It should be understood that since the training Mel spectrum is subjected to sound source separation processing, there are many ways to obtain the training sub-spectrum, so the embodiment of the present application is not limited to separating the training sub-spectrum through the above-mentioned specific embodiments.

[0086] In step S304 of some embodiments of the present application, the audio processing model is iteratively trained based on the training sub-spectrum and the music genre label. It should be noted that after the audio sample is subjected to audio sampling and sound source separation processing to obtain the training sub-spectrum, the audio processing model can be iteratively trained based on the training sub-spectrum and the music genre label. It should be emphasized that a musical work is generated by superimposing multiple types of tracks in a specific time interval after arrangement. Therefore, after audio sampling and sound source separation processing, each audio sample can obtain several training sub-spectra, wherein each training sub-spectra is used to characterize a type of sound in the audio sample. It should be clear that the category features of each music genre are significantly correlated with the superimposed features of each track in the music work. Therefore, as the iterative training continues, the audio processing model is gradually able to distinguish the features presented by each music genre category on the training sub-spectrum through continuous updating, and the ability of the audio processing model to identify music genres is also improved accordingly.

[0087] Pre-training the audio processing model via the method shown in step S301 to step S304 can improve the music genre recognition ability of the audio processing model. In some more specific embodiments of the present application, the audio processing model may include multiple feature extraction modules, a sound track feature splicing module, and a fully connected layer classifier. It should be noted that the audio processing model is iteratively trained based on the training sub-spectra and the music genre labels, and the training sub-spectra of each separated sound can be first input into the feature extraction module for feature extraction processing. In some embodiments, multiple feature extraction modules have the same structure but do not share parameters. The feature extraction module may include a two-dimensional convolutional neural network and a bidirectional long short-term memory network. In order to downsample the features in some embodiments, a two-dimensional maximum pooling layer may also be added after each convolution layer of the two-dimensional convolutional neural network. After the convolutional neural network extracts the spectral features of the training sub-spectrum of each sound, it can further extract the temporal features of the training sub-spectrum based on the bidirectional long short-term memory network. Each feature extraction module then outputs the extracted training audio features to the sound track feature splicing module. After the sound track feature splicing module receives the training audio features corresponding to each sound in the audio sample, it splices multiple training audio features to obtain a total training feature vector. Further, this total training feature vector is input into the fully connected layer classifier for classification. The final classification prediction result is called training classification data. It should be pointed out that the fully connected layer classifier can be a Sigmoid fully connected classifier or other types of classifiers, which are not given one by one here. After obtaining the training classification data, the training classification data is compared with the music genre label to clarify the recognition accuracy of this round of iterative training. After this round of iterative training is completed, the audio processing model is updated based on the training classification data. Since the category characteristics of each music genre are significantly correlated with the overlapping features of each track in the music work, as the iterative training continues, the audio processing model is continuously updated and gradually able to distinguish the characteristics of each music genre category presented in the training sub-spectrum, and the audio processing model's ability to recognize music genres is also improved accordingly.

[0088] Reference Figure 4 According to the music genre identification method in some embodiments of the present application, step S304 may include, but is not limited to, the following steps S401 to S403.

[0089] Step S401, in each round of iterative training, the training sub-spectrum is classified and identified based on the audio processing model to obtain training classification data;

[0090] Step S402, performing a difference comparison based on the training classification data and the music genre label to obtain classification deviation data;

[0091] Step S403: after each round of iterative training, the model parameters of the audio processing model are updated based on the classification deviation data.

[0092] In steps S401 to S402 of some embodiments of the present application, in each round of iterative training, the training sub-spectra are first classified and identified based on the audio processing model to obtain training classification data, and then the difference between the training classification data and the music genre label is compared to obtain classification deviation data. It should be noted that in each round of iterative training, the audio processing model first extracts features from multiple training sub-spectra of the same training Mel spectrum, and then concatenates the extracted multiple training audio features to obtain a total training feature vector. Further, this total training feature vector is input into the fully connected layer classifier for classification, and the final classification prediction result is called training classification data. It should be emphasized that, precisely because the training classification data reflects the classification prediction results of the audio processing model in the current iteration round, the classification deviation data is obtained based on the difference comparison between the training classification data and the music genre label, wherein the classification deviation data is used to reflect the music genre recognition ability of the audio processing model in the current iteration round. If the classification deviation data is larger, it means that the difference between the training classification data and the music genre label is larger, and if the classification deviation data is smaller, it means that the difference between the training classification data and the music genre label is smaller. It should be pointed out that the types of classification deviation data are various, which can be the output value of the loss function calculated based on the training classification data and the music genre label, or it can be the similarity obtained by processing the training classification data and the music genre label based on various similarity algorithms. The classification deviation data can also be other types, not limited to the specific embodiments listed above.

[0093] Step S403, after each round of iterative training, the model parameters of the audio processing model are updated based on the classification deviation data. It should be noted that there are generally two types of parameters in the neural network model: one type of parameter is the tuning parameters in the machine learning algorithm, which need to be flexibly set according to existing or existing experience, also known as hyperparameters. For example, the regularization coefficient λ, the depth of the tree in the decision tree model. Hyperparameters are also a type of parameter, which has the characteristics of parameters, such as unknown, that is, it is not a known constant, but a manually configurable value, and it needs to be assigned a "correct" value based on existing or existing experience, that is, a flexibly set value, which is not obtained through system learning; the other type of parameter needs to be learned and estimated from the data, called model parameters, that is, the learnable parameters of the model itself. For example, the weighting coefficient (slope) and the deviation term (intercept) of the linear regression line are both model parameters. Learnable parameters specifically refer to parameter values ​​learned during the training process of the neural network model. For learnable parameters, they usually start from a set of random values, and then update these values ​​in an iterative manner as the neural network model learns. In fact, when the "neural network model is learning", it is more accurate to say that the parameters of the neural network model are in the process of iterative update, gradually determining the appropriate values ​​of these parameters. It should be noted that the so-called appropriate value can be a value that minimizes or converges the loss function. After each round of iterative training, the model parameters of the audio processing model need to be updated based on the classification deviation data so that the model parameters are adjusted to appropriate values.

[0094] Through the iterative training process shown from step S401 to step S403, the model parameters can be continuously updated during the iterative training process to enhance the music genre recognition capability of the audio processing model during the pre-training process.

[0095] Reference Figure 5 According to the music genre identification method of some embodiments of the present application, the classification deviation data is the gradient value of the loss function, and step S403 may include but is not limited to the following steps S501 to S502.

[0096] Step S501, determining whether the loss function gradient value meets the second preset condition;

[0097] Step S502: When the loss function gradient value meets the second preset condition, the weight and bias of the audio processing model are adjusted based on the loss function gradient value to update the audio processing model.

[0098] In steps S501 to S502 of some embodiments of the present application, it is first determined whether the gradient value of the loss function meets the second preset condition. When the gradient value of the loss function meets the second preset condition, the weight and bias of the audio processing model are adjusted based on the gradient value of the loss function to update the audio processing model. It should be noted that the gradient value of the loss function is used to define whether the loss function in the audio processing model converges or is the minimum value in the current iteration round. If the loss function in the audio processing model does not converge or is not the minimum value in the current iteration round, it meets the second preset condition. At this time, the weight and bias of the audio processing model can be adjusted based on the gradient value of the loss function to update the audio processing model and start the next round of iterative training; if the loss function in the audio processing model converges or is the minimum value in the current iteration round, it does not meet the second preset condition. At this time, the audio processing model of the current iteration round can be output, and the iterative training is completed to obtain the pre-trained audio processing model.

[0099] In some exemplary embodiments of the present application, the audio processing model needs to be optimized, where optimization refers to the task of minimizing the loss function. The gradient is the derivative of the loss function of the data classification model under the current weights and biases, and the loss function is used to measure the gap between the output of the neural network and the expected output. Gradient descent refers to calculating a gradient on each weight in the neural network for each training element. Since the neural network does not output the expected value of the training element, the gradient of each weight will give you a hint on how to modify the weight to achieve the expected output. If the neural network does output the expected result, the gradient of each weight will be 0, indicating that there is no need to modify the weight. The gradient is essentially the partial derivative of the loss function for each weight in the neural network. Each weight has a gradient, which is the slope of the loss function, and the weight is the connection between two neurons. Calculating the gradient of the loss function can determine whether the training algorithm should increase or decrease the weight. Similarly, the determination of the gradient helps to reduce the error of the neural network. The error is the difference between the expected output and the actual output of the neural network. In general, the gradient tells the neural network the following information: zero gradient-the weight will not cause the error of the neural network; negative gradient-the weight should be increased to reduce the error; positive gradient-the weight should be reduced to reduce the error.

[0100] The embodiment of the present application shown in step S501 to step S502 can update the audio processing model in iterative training based on the gradient value of the loss function to enhance the music genre recognition capability of the audio processing model during pre-training.

[0101] In step S305 of some embodiments of the present application, when the audio processing model meets the first preset condition in the iterative training, the trained audio processing model is obtained. It should be noted that the first preset condition refers to the condition used to define whether the music genre recognition ability of the audio processing model meets the expected requirements. Therefore, when the audio processing model meets the first preset condition in the iterative training, it can be determined that the audio processing model has reached the expected requirements in the iterative training. At this time, the audio processing model of the current iteration round can be output to obtain the trained audio processing model. It should be pointed out that the first preset condition will also be different in different iterative training scenarios. For example, the first preset condition can be whether the output value of the loss function in the current iteration round converges to a certain value, or it can be whether the gradient value of the loss function is close to zero, or it can be whether the recognition accuracy reaches a preset threshold in the current iteration round. It should be clear that the types of audio processing models that meet the first preset condition in iterative training are various, and can include, but not limited to the specific embodiments listed above.

[0102] Reference Figure 6 According to the music genre identification method of some embodiments of the present application, the classification deviation data is the output value of the loss function, and step S305 may include but is not limited to the following steps S601 to S602.

[0103] Step S601, when the output value of the loss function is less than a preset threshold, it is determined that the audio processing model meets the first preset condition in iterative training;

[0104] Step S602: output the audio processing model in the current round to obtain a trained audio processing model.

[0105] In steps S601 to S602 of some embodiments of the present application, when the output value of the loss function is less than the preset threshold, it is determined that the audio processing model meets the first preset condition in the iterative training, and then the audio processing model under the current round is output to obtain the trained audio processing model. It should be noted that during the model training process, there will be a difference between the model prediction value (estimated value) and the actual value (expected value, reference value, ground truth), and "loss" means the penalty imposed on the model for failing to produce the expected result. The loss function is used to determine the performance of the model and find the optimization direction by comparing the predicted output and the expected output of the model. If the deviation between the predicted output and the expected output is very large, the output value of the loss function will be very large; if the deviation is very small or the value is almost the same, the output value of the loss function will be very low. Therefore, during the iterative training process, it is necessary to continuously adjust the model parameters through a suitable loss function when the model is trained on the data set until the model parameters are adjusted to an appropriate value, wherein the appropriate value may be a value that minimizes or converges the loss function. Therefore, in some embodiments of the present application, the first preset condition is configured according to whether the output value of the loss function is less than the preset threshold. When the audio processing model meets the first preset condition in the iterative training, that is, the output value of the loss function is less than the preset threshold, it can be determined that the audio processing model's ability to recognize music genres in the iterative training has reached the expected requirements. At this time, the audio processing model of the current iteration round can be output to obtain the trained audio processing model to enhance the music genre recognition ability of the audio processing model during the pre-training process.

[0106] In step S104 of some embodiments of the present application, the target sub-spectrum is input into the pre-trained audio processing model for genre recognition processing to obtain the music genre category corresponding to the target audio file. It should be noted that the present application embodiment is pre-trained for the audio processing model, and its purpose is to enhance the ability of the audio processing model to identify music genres. When the audio processing model has been pre-trained, it already has the ability to identify music genres. At this time, the target sub-spectrum is input into the pre-trained audio processing model for genre recognition processing, and the music genre category corresponding to the target audio file can be obtained based on the ability of the audio processing model to identify music genres. It should be emphasized that because the music work is generated by the superposition of multiple types of tracks in a specific time interval after arrangement, the music work will have the difference between the complexity and simplicity of the arrangement. And the category features of each music genre are significantly related to the superposition features of each track in the music work. In some embodiments of this scheme, the target sub-spectrum is obtained by separating these tracks, and then the target sub-spectrum is subjected to genre recognition processing through the pre-trained audio processing model, so as to achieve more efficient music genre recognition.

[0107] Reference Figure 7 According to the music genre recognition method of some embodiments of the present application, the audio processing model includes a feature extraction module, a sound track feature splicing module and a fully connected layer classifier, and step S104 may include but is not limited to the following steps S701 to S703.

[0108] Step S701, extracting features from each target sub-spectrum based on a feature extraction module to obtain a plurality of audio feature sub-vectors;

[0109] Step S702, based on the sound track feature splicing module, multiple audio feature sub-vectors are spliced ​​to obtain an audio feature total vector;

[0110] Step S703: Perform genre classification processing on the total audio feature vector based on the fully connected layer classifier to obtain the music genre category corresponding to the target audio file.

[0111] In steps S701 to S703 of some embodiments of the present application, each target sub-spectrum is first feature extracted based on the feature extraction module to obtain multiple audio feature sub-vectors, and then the multiple audio feature sub-vectors are spliced ​​based on the sound track feature splicing module to obtain the total audio feature vector, and then the total audio feature vector is subjected to genre classification based on the fully connected layer classifier to obtain the music genre category corresponding to the target audio file. In some more specific embodiments of the present application, the audio processing model may include multiple feature extraction modules, sound track feature splicing modules, and fully connected layer classifiers. It should be noted that the target sub-spectrum is input into the pre-trained audio processing model for genre recognition processing, and the target sub-spectrum of each separated sound can be first input into the feature extraction module for feature extraction processing. In some embodiments, multiple feature extraction modules have the same structure, but do not share parameters, and the feature extraction module may include a two-dimensional convolutional neural network and a bidirectional long short-term memory network. In order to downsample the features, some embodiments may also add a two-dimensional maximum pooling layer after each convolution layer of the two-dimensional convolutional neural network. After the convolutional neural network extracts the spectral features of the target sub-spectrum of each sound, it can further extract the temporal features of the target sub-spectrum based on the bidirectional long short-term memory network to obtain multiple audio feature sub-vectors. Each feature extraction module then outputs the extracted audio feature sub-vectors to the sound track feature splicing module. After the sound track feature splicing module receives the audio feature sub-vectors corresponding to each sound in the audio sample, it splices the multiple audio feature sub-vectors to obtain a total target feature vector, that is, the total audio feature vector. Further, the total audio feature vector is input into the fully connected layer classifier, and the genre classification processing of the total audio feature vector is performed based on the fully connected layer classifier to obtain the music genre category corresponding to the target audio file. It should be pointed out that the fully connected layer classifier can be a Sigmoid fully connected classifier or other types of classifiers, which are not given one by one here. Since the category characteristics of each music genre are significantly correlated with the overlapping characteristics of each audio track in the music work, and the audio processing model has been pre-trained to have the ability to recognize music genres, the pre-trained audio processing model can distinguish the characteristics of each music genre category on the target sub-spectrum, thereby accurately identifying the music genre category corresponding to the target audio file, thereby improving the accuracy of music genre recognition.

[0112] Figure 8 The electronic device 800 provided in the embodiment of the present application is shown. The electronic device 800 includes: a processor 801, a memory 802, and a computer program stored in the memory 802 and executable on the processor 801, and the computer program is used to execute the above-mentioned music genre identification method when it is executed.

[0113] The processor 801 and the memory 802 may be connected via a bus or other means.

[0114] The memory 802 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs and non-transitory computer executable programs, such as the music genre identification method described in the embodiment of the present application. The processor 801 implements the above-mentioned music genre identification method by running the non-transitory software program and instructions stored in the memory 802.

[0115] The memory 802 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function. The data storage area may store and execute the above-mentioned music genre recognition method. In addition, the memory 802 may include a high-speed random access memory 802, and may also include a non-transitory memory 802, such as at least one storage device storage device, a flash memory device or other non-transitory solid-state storage device. In some embodiments, the memory 802 may optionally include a memory 802 remotely arranged relative to the processor 801, and these remote memories 802 may be connected to the electronic device 800 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0116] The non-transient software program and instructions required to implement the above-mentioned music genre identification method are stored in the memory 802. When executed by one or more processors 801, the above-mentioned music genre identification method is executed, for example, Figure 1 Steps S101 to S104 of the method, Figure 2 Steps S201 to S202 of the method, Figure 3 Steps S301 to S305 of the method, Figure 4 Steps S401 to S403 of the method, Figure 5 Steps S501 to S502 of the method, Figure 6 Steps S601 to S602 of the method, Figure 7 The method steps S701 to S703 in FIG.

[0117] The embodiment of the present application also provides a computer-readable storage medium storing computer-executable instructions, and the computer-executable instructions are used to execute the above-mentioned music genre identification method.

[0118] In one embodiment, the computer readable storage medium stores computer executable instructions, which are executed by one or more control processors, for example, Figure 1 Steps S101 to S104 of the method, Figure 2 Steps S201 to S202 of the method, Figure 3 Steps S301 to S305 of the method, Figure 4 Steps S401 to S403 of the method, Figure 5 Steps S501 to S502 of the method, Figure 6 Steps S601 to S602 of the method, Figure 7 The method steps S701 to S703 in FIG.

[0119] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0120] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, storage device storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. In addition, it is well known to those skilled in the art that communication media generally include computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium. It should also be understood that the various implementations provided in the embodiments of the present application can be arbitrarily combined to achieve different technical effects.

[0121] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the above implementation mode. Technical personnel familiar with the field can also make various equivalent modifications or substitutions under the shared conditions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A method for identifying a music genre, characterized in that: include: Obtain the target audio source file to be identified; Performing audio sampling processing on the target sound source file to obtain a target Mel spectrum; Performing sound source separation processing on the target Mel spectrum to obtain a target sub-spectrum, where the target sub-spectrum is used to characterize a type of sound in the target sound source file; The target sub-spectrum is input into a pre-trained audio processing model for genre recognition processing to obtain the music genre category corresponding to the target audio file.

2. The method according to claim 1, characterized in that The performing sound source separation processing on the target Mel spectrum to obtain a target sub-spectrum includes: The target Mel spectrum is input into a pre-trained sound source separation model for sound source separation processing to obtain a plurality of target sub-spectra.

3. The method according to claim 2, characterized in that The audio processing model includes a feature extraction module, a sound track feature splicing module and a fully connected layer classifier. The target sub-spectrum is input into a pre-trained audio processing model for genre recognition processing to obtain a music genre category corresponding to the target audio file, including: Performing feature extraction on each of the target sub-spectrums based on the feature extraction module to obtain a plurality of audio feature sub-vectors; Based on the sound track feature splicing module, a plurality of the audio feature sub-vectors are spliced ​​to obtain an audio feature total vector; Based on the fully connected layer classifier, genre classification processing is performed on the total audio feature vector to obtain a music genre category corresponding to the target audio file.

4. The method according to claim 1, characterized in that The performing audio sampling processing on the target sound source file to obtain a target Mel spectrum includes: Obtaining a preset spectrum sampling rate, a preset Fourier transform window length, a preset window step size, and a preset number of Mel filters; Based on the spectrum sampling rate, the Fourier transform window length, the window step size and the number of Mel filters, a Mel feature extraction function is called to perform audio sampling on the target sound source file to obtain the target Mel spectrum.

5. The method according to claim 1, characterized in that Before inputting the target sub-spectrum into a pre-trained audio processing model for genre recognition processing to obtain the music genre category corresponding to the target audio file, the method further includes pre-training the audio processing model, specifically including: Acquire a training data set, wherein the training data set includes a plurality of audio samples, each of the audio samples is configured with a corresponding music genre label; Performing audio sampling processing on the audio sample to obtain a training Mel spectrum; Performing sound source separation processing on the training Mel spectrum to obtain a training sub-spectrum, wherein the training sub-spectrum is used to characterize a type of sound in the audio sample; Iteratively training the audio processing model based on the training sub-spectrum and the music genre label; When the audio processing model meets the first preset condition in iterative training, the trained audio processing model is obtained.

6. The method according to claim 5, characterized in that The iterative training of the audio processing model based on the training sub-spectrum and the music genre label comprises: In each round of the iterative training, the training sub-spectrum is classified and identified based on the audio processing model to obtain training classification data; Comparing the difference between the training classification data and the music genre label to obtain classification deviation data; After each round of iterative training, the model parameters of the audio processing model are updated based on the classification deviation data.

7. The method according to claim 6, characterized in that The classification deviation data is a loss function gradient value, and updating the model parameters of the audio processing model based on the classification deviation data includes: When the loss function gradient value meets the second preset condition, the weight and bias of the audio processing model are adjusted based on the loss function gradient value to update the audio processing model.

8. The method according to claim 6, characterized in that The classification deviation data is the output value of the loss function, and when the audio processing model meets the first preset condition in the iterative training, the trained audio processing model is obtained, including: When the output value of the loss function is less than a preset threshold, it is determined that the audio processing model meets the first preset condition in iterative training, and the trained audio processing model is obtained.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program, and the processor implements the music genre identification method according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the music genre identification method according to any one of claims 1 to 8.

Images