Artist classification method, system, terminal and storage medium

Abstract

The present invention discloses an artist classification method, system, terminal, and storage medium. The method comprises: obtaining a first spectrum of an original audio signal, encoding the first spectrum using an encoder to obtain an encoded continuous variable, wherein the continuous variable contains timbre information and audio content information of the original audio signal; performing instance regularization on the continuous variable, and vector quantizing the instance regularization result using discrete codewords in a codebook, wherein the vector quantization result represents the audio content information of the original audio signal; subtracting the continuous variable from the vector quantization result to obtain the difference between the continuous variable and the vector quantization result, and using the difference as a first timbre feature of the original audio signal, wherein the first timbre feature is a frame-by-frame variation feature of the timbre in the original audio signal; inputting the first timbre feature into an artist classification network, and outputting an artist classification result of the original audio signal through the artist classification network. This application improves the accuracy of artist classification.

Description

Technical Field

[0001] The present invention relates to the technical field of audio signal recognition, and in particular to an artist classification method, system, terminal and storage medium. Background Art

[0002] Artist classification involves identifying the artist in a given song clip. Accurately classifying artists is crucial for audio-based music retrieval, such as automatic tagging, recommending similar artists, and managing music libraries by artist. For familiar songs, it's often easy to identify the artist simply by hearing a few words or a line of lyrics. However, for many unknown songs, some artists' voices are difficult to distinguish, making it difficult for people to accurately identify the artist without being familiar with the song.

[0003] Existing artist classification methods typically employ song identification techniques. For example, they employ database and audio fingerprinting technologies to extract audio fingerprints from existing songs and store them in a database. Then, they extract audio fingerprints from the songs to be identified and match them with the audio fingerprints stored in the database to complete the artist classification task. This method requires the establishment of a massive music library, which is resource-intensive and expensive to build. Another approach uses deep learning models to extract features and distinguish between different singers. While this approach avoids the need to build a massive music library, it suffers from low artist classification accuracy because the deep learning model learns the differences between the entire song rather than the vocal timbre of the singer. While artist classification should primarily focus on the singer's voice, not the song itself, it can be quite complex. Summary of the Invention

[0004] The present invention provides an artist classification method, system, terminal and storage medium, aiming to solve technical problems of existing artist classification methods such as resource consumption, high construction cost and low artist classification accuracy.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A method for classifying artists, comprising:

[0007] Obtaining a first spectrum of the original audio signal, and encoding the first spectrum using an encoder to obtain an encoded continuous variable V, wherein the continuous variable V contains timbre information and audio content information of the original audio signal;

[0008] Performing instance regularization on the continuous variable V, and performing vector quantization on the instance regularization result using discrete codewords in a codebook to obtain a vector quantization result, wherein the vector quantization result represents audio content information of the original audio signal;

[0009] Subtracting the continuous variable from the vector quantization result to obtain a difference between the continuous variable and the vector quantization result, and using the difference as a first timbre feature of the original audio signal, where the first timbre feature is a frame-by-frame variation feature of the timbre in the original audio signal;

[0010] The first timbre feature is input into an artist classification network, and the artist classification result of the original audio signal is output through the artist classification network.

[0011] The technical solution adopted in the embodiment of the present invention further includes: obtaining the first spectrum of the original audio signal includes:

[0012] The original audio signal is input into the pre-trained vocal accompaniment separation model, and the vocal dry sound in the original audio signal is extracted by the vocal accompaniment separation model;

[0013] Perform short-time Fourier transform on the extracted dry vocal sound to obtain a first spectrum X of the original audio signal.

[0014] The technical solution adopted in the embodiment of the present invention further includes: performing instance regularization on the continuous variable V includes:

[0015] For each continuous variable V, calculate its mean u and variance σ^;

[0016] Normalize the continuous variable V according to the mean u and variance σ^ to obtain an instance regularization result V';

[0017] The normalization formula is: where ε is the bias term.

[0018] The technical solution adopted by the embodiment of the present invention further includes: the vector quantization of the instance regularization result using the discrete codewords in the codebook includes:

[0019] A codeword search is performed on the instance regularization result V', the distance between the instance regularization result V' and each codeword in the codebook is calculated, and argmin is calculated to obtain the codeword with the closest distance to represent the instance regularization result V'.

[0020] The technical solution adopted in the embodiment of the present invention further includes: subtracting the continuous variable from the vector quantization result to obtain the difference between the continuous variable and the vector quantization result further includes:

[0021] The difference result between the continuous variable and the vector quantization result is averaged frame by frame, and replicated and expanded according to the number of frames to obtain the second timbre feature of the original audio signal, where the second timbre feature indicates that the timbre of the same singer does not change over time.

[0022] The technical solution adopted by the embodiment of the present invention further includes: before inputting the first timbre feature into the artist classification network, the following steps are further included:

[0023] fusing the vector quantization result with the second timbre feature, and inputting the fusion result into a decoder for spectrum reconstruction, wherein the spectrum reconstruction task is used to constrain the first timbre feature to be a frame-by-frame timbre feature;

[0024] Inputting the fusion result into the decoder for spectrum reconstruction includes:

[0025] Based on the fusion result of the vector quantization result and the second timbre feature, a decoder is used to perform spectrum reconstruction to obtain a reconstructed second spectrum X';

[0026] A spectrum reconstruction task is completed by calculating the MAE loss for the first spectrum X and the second spectrum X'. The vector quantization result and the second timbre feature are constrained by the spectrum reconstruction task so that the vector quantization result contains as much audio content information as possible, the second timbre feature contains as much timbre information as possible, and the first timbre feature is constrained to be a frame-by-frame timbre feature.

[0027] The technical solution adopted by the embodiment of the present invention further includes: inputting the first timbre feature into an artist classification network, and outputting an artist classification result of the original audio signal through the artist classification network includes:

[0028] The artist classification network includes a 2D convolutional layer, a GRU-RNN layer and a fully connected layer. The 2D convolutional layer is used to learn the features of the first timbre feature in the spatial dimension, and the GRU-RNN network is used to learn the features of the first timbre feature in the time domain. It is connected to the category through the fully connected layer for softmax classification and recognition, and outputs the singer label corresponding to the original audio signal.

[0029] Another technical solution adopted by an embodiment of the present invention is: an artist classification system, comprising:

[0030] Coding calculation module: used to obtain a first spectrum of the original audio signal and encode the first spectrum using an encoder to obtain an encoded continuous variable V, wherein the continuous variable V contains the timbre information and audio content information of the original audio signal;

[0031] A vector quantization module is configured to perform instance regularization on the continuous variable V and perform vector quantization on the instance regularization result using discrete codewords in a codebook to obtain a vector quantization result, wherein the vector quantization result represents audio content information of the original audio signal;

[0032] A timbre acquisition module is configured to perform a subtraction between the continuous variable and the vector quantization result to obtain a difference between the continuous variable and the vector quantization result, and use the difference as a first timbre feature of the original audio signal, wherein the first timbre feature is a frame-by-frame variation feature of the timbre in the original audio signal;

[0033] Artist classification module: used to input the first timbre feature into the artist classification network, and output the artist classification result of the original audio signal through the artist classification network.

[0034] Another technical solution adopted by an embodiment of the present invention is: a terminal, the terminal including a processor and a memory coupled to the processor, wherein:

[0035] The memory stores program instructions for implementing the above-mentioned artist classification method;

[0036] The processor is configured to execute the program instructions stored in the memory to perform the artist classification operation.

[0037] Another technical solution adopted by an embodiment of the present invention is: a storage medium storing program instructions executable by a processor, wherein the program instructions are used to execute the above-mentioned artist classification method.

[0038] The artist classification method, system, terminal, and storage medium of the embodiments of the present invention represent audio content information by introducing a vector quantization method, and use discrete codeword variable representation to reduce the data volume of continuous variables, thereby better reflecting the timbre information of the audio signal. By fusing the timbre information and audio content information and performing spectrum reconstruction, accurate decoupling of the audio content information and the timbre information is achieved, and the timbre information is constrained to frame-by-frame timbre features. Finally, artist classification is performed based on the frame-by-frame timbre features, thereby enhancing the classification and recognition of the frame-by-frame timbre features in multiple dimensions and improving the accuracy of artist classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flowchart of the artist classification method according to the first embodiment of the present invention;

[0040] Figure 2 is a flowchart of an artist classification method according to a second embodiment of the present invention;

[0041] Figure 3 is a schematic diagram of the structure of an artist classification system according to an embodiment of the present invention;

[0042] Figure 4 is a schematic diagram of the terminal structure of an embodiment of the present invention;

[0043] Figure 5 It is a schematic diagram of the storage medium structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0045] The terms "first", "second" and "third" in the present invention are used only for descriptive purposes and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, a feature defined as "first", "second" and "third" may explicitly or implicitly include at least one of such features. In the description of the present invention, "multiple" means at least two, for example, two, three, etc., unless otherwise clearly and specifically defined. All directional indications in the embodiments of the present invention (such as up, down, left, right, front, back...) are only used to explain the relative positional relationship, movement, etc. between the components under a specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0046] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0047] See also Figure 1 , is a flow chart of the artist classification method according to the first embodiment of the present invention. The artist classification method according to the first embodiment of the present invention comprises the following steps:

[0048] S100: Obtain a first spectrum of an original audio signal, and encode the first spectrum using an encoder to obtain an encoded continuous variable V, where the continuous variable V contains timbre information and audio content information of the original audio signal;

[0049] S101: performing instance regularization on the continuous variable V, and performing vector quantization on the instance regularization result using discrete codewords in the codebook to obtain a vector quantization result, where the vector quantization result represents audio content information of the original audio signal;

[0050] S102: Subtracting the continuous variable from the vector quantization result to obtain a difference between the continuous variable and the vector quantization result, and using the difference as a first timbre feature of the original audio signal;

[0051] S103: Input the first timbre feature into an artist classification network, and output an artist classification result of the original audio signal through the artist classification network.

[0052] See also Figure 2 , is a flow chart of an artist classification method according to a second embodiment of the present invention. The artist classification method according to the second embodiment of the present invention comprises the following steps:

[0053] S200: Inputting the original audio signal into a pre-trained vocal accompaniment separation model, and extracting the dry vocal sound from the original audio signal through the vocal accompaniment separation model;

[0054] In this step, the vocal accompaniment separation model uses a U-net network structure consisting of a convolutional network, a deconvolutional network, and corresponding skip links in the intermediate layers. After pre-training the U-net-based vocal accompaniment separation model to obtain the optimal parameters, the raw vocal sound is extracted from the input audio signal.

[0055] S201: performing a short-time Fourier transform (STFT) on the extracted dry human voice to obtain a first spectrum X of the original audio signal;

[0056] In this step, the STFT algorithm uses a window function to perform a frame operation on the human voice dry sound signal, and performs an FFT (Fast Fourier Transform) on each frame signal after framing and windowing to obtain the frequency spectrum of each frame signal.

[0057] S202: Input the first spectrum X into an encoder, and use the encoder to encode the first spectrum X to obtain an encoded continuous variable V, where the continuous variable V contains timbre information and audio content information of the original audio signal;

[0058] In this step, the encoder includes four 2D convolutional layers and two LSTM (Long Short-Term Memory) layers. The input first spectrum X is processed through the 2D convolutional layers to obtain spatial features of size 256*80. The LSTM layer then learns the temporal features of the first spectrum X to obtain a continuous variable V of dimension 80.

[0059] S203: Perform instance regularization on the continuous variable V using the instance regularization layer to obtain an instance regularization result V';

[0060] In this step, the instance regularization of the continuous variable V is specifically as follows: for each continuous variable V, its mean u and variance σ^ are calculated respectively, and then the continuous variable V is normalized according to the mean u and variance σ^ to obtain the instance regularization result V'. The normalization formula is: where ε is the bias term.

[0061] S204: performing codeword search on the instance regularization result V', and performing vector quantization on the instance regularization result V' using discrete codewords in the codebook to obtain a vector quantization result C. The vector quantization result C only represents audio content information in the original audio signal;

[0062] In this step, when performing a codeword search, if there are multiple codewords in the codebook, the distance between the instance-normalized result V' and each codeword in the codebook is calculated, and the argmin is calculated to obtain the codeword with the closest distance to represent the instance-normalized result V'. Vector quantization (VQ) is a lossy data compression method based on block coding rules. It compresses data without much information loss by grouping several scalar data into a vector and then quantizing the entire vector space. The codebook has a fixed size, that is, the number of codewords is fixed. To represent different instance-normalized results V', the codeword variables in the codebook need to be adjusted. By continuously adjusting the codebook during the training process, the codewords that are finally fixed are the trained discrete variables. The process of representing the instance-normalized result V' with discrete codeword variables is called vector quantization. For example, if the instance-normalized result V' is 1.1 or 1.2, it can be represented as 1 in the codebook, while if the instance-normalized result V' is 2.1, 2.2, or 1.9, it can be represented as 2 in the codebook. Based on this, the embodiment of the present application introduces a vector quantization method to represent the audio content information, thereby reducing the data volume of the continuous variable V.

[0063] S205: Subtracting the continuous variable V from the vector quantization result C to obtain a difference between the continuous variable and the discrete variable, and using the difference as the first timbre feature S' of the original audio signal. The subtraction result is then averaged frame by frame and replicated and expanded according to the number of frames to obtain a second timbre feature S of the original audio signal.

[0064] In this step, the difference between the vector quantization result C and the continuous variable V reflects the timbre characteristics of the audio signal. The difference result is averaged frame by frame, calculating the timbre mean over all time periods. The first timbre feature S' captures the detailed frame-by-frame timbre variations in the original audio signal, while the second timbre feature S indicates that the timbre of the same singer remains constant over time.

[0065] S206: Fusing the vector quantization result C with the second timbre feature S, and inputting the fusion result into the decoder for spectrum reconstruction. The spectrum reconstruction task is used to constrain the first timbre feature S' to be a frame-by-frame timbre feature.

[0066] In this step, based on the fusion of the vector quantization result C and the second timbre feature S, a decoder is used to perform spectrum reconstruction, obtaining a reconstructed second spectrum X'. The spectrum reconstruction task is completed by calculating the mean absolute error (MAE) loss between the first spectrum X and the reconstructed second spectrum X', thereby achieving accurate decoupling of audio content information and timbre information. The spectrum reconstruction task constrains the vector quantization result C and the second timbre feature S, ensuring that the vector quantization result C contains as much audio content information as possible, while the second timbre feature S contains as much timbre information as possible. At the same time, the first timbre feature S' is constrained to be a frame-by-frame timbre feature.

[0067] S207: Inputting the frame-by-frame timbre features into an artist classification network, and outputting an artist classification result of the original audio signal through the artist classification network;

[0068] In this step, the artist classification network adopts a CRNN (Convolutional Recurrent Neural Network) structure, including 4 layers of 2D convolutional layers, 2 layers of GRU-RNN layers, and a final fully connected layer. First, the 2D convolutional layers are used to learn the spatial characteristics of the frame-by-frame timbre features. Then, the GRU-RNN network is used to learn the temporal characteristics of the frame-by-frame timbre features, enhancing the multi-dimensional classification and recognition of the frame-by-frame timbre features. Finally, the fully connected layer is used to connect to the category for softmax classification and recognition, and output the singer label corresponding to the original audio signal. The embodiment of the present application enhances the multi-dimensional classification and recognition of the frame-by-frame timbre features and improves the accuracy of artist classification by performing artist classification based on the frame-by-frame timbre features.

[0069] Based on the above, the artist classification method of the second embodiment of the present invention represents audio content information by introducing a vector quantization method, and uses discrete codeword variables to represent the reduced data volume of continuous variables, thereby better reflecting the timbre information of the audio signal; by fusing the timbre information and the audio content information and performing spectrum reconstruction, accurate decoupling of the audio content information and the timbre information is achieved, and the timbre information is constrained to frame-by-frame timbre features. Finally, artist classification is performed based on the frame-by-frame timbre features, thereby enhancing the classification and recognition of the frame-by-frame timbre features in multiple dimensions and improving the accuracy of artist classification.

[0070] In an optional embodiment, it is also possible to upload the results of the artist classification method to the blockchain.

[0071] Specifically, summary information is generated based on the results of the artist classification method. Specifically, the summary information is generated by hashing the results of the artist classification method, for example, using the sha256s algorithm. Uploading the summary information to the blockchain ensures its security and fairness and transparency to users. Users can download the summary information from the blockchain to verify whether the results of the artist classification method have been tampered with. The blockchain referred to in this example is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a string of data blocks generated using cryptographic methods. Each data block contains information about a batch of network transactions, which is used to verify the validity of the information (to prevent counterfeiting) and generate the next block. Blockchain can include the blockchain underlying platform, the platform product service layer, and the application service layer.

[0072] See also Figure 3 , is a schematic diagram of the structure of an artist classification system according to an embodiment of the present invention. The artist classification system 40 according to an embodiment of the present invention includes:

[0073] Coding calculation module 41: used to obtain the first spectrum of the original audio signal, and use the encoder to encode the first spectrum to obtain the encoded continuous variable V, where the continuous variable V contains the timbre information and audio content information of the original audio signal; wherein, the embodiment of the present application inputs the original audio signal into a pre-trained vocal accompaniment separation model, extracts the human voice dry sound in the original audio signal through the human voice accompaniment separation model, performs a short-time Fourier transform on the extracted human voice dry sound, obtains the first spectrum X of the original audio signal, and inputs the first spectrum X into the encoder, the encoder includes 4 layers of 2D convolution layers and 2 layers of LSTM layers. After the input first spectrum X passes through the 2D convolution layer, a spatial feature of a size of 256*80 is obtained, and then the time series feature of the first spectrum X is learned through the LSTM layer to obtain a continuous variable V with a dimension of 80.

[0074] Vector quantization module 42: used to perform instance regularization on the continuous variable V and vector quantize the instance regularization result using discrete codewords in the codebook to obtain a vector quantization result, which represents the audio content information of the original audio signal. The instance regularization of the continuous variable V is specifically as follows: for each continuous variable V, its mean u and variance σ^ are calculated respectively, and then the continuous variable V is normalized according to the mean u and variance σ^ to obtain the instance regularization result V'. The normalization formula is: Where ε is the bias term. The vector quantization process includes: performing a codeword search on the instance-normalized result V', calculating the distance between the instance-normalized result V' and each codeword in the codebook, and then taking the argmin to obtain the codeword with the closest distance to represent the instance-normalized result V'.

[0075] Timbre acquisition module 43: used to subtract the continuous variable from the vector quantization result to obtain the difference between the continuous variable and the vector quantization result, and use the difference as the first timbre feature of the original audio signal; at the same time, the subtraction result of the continuous variable and the vector quantization result is averaged frame by frame, and replicated and expanded according to the number of frames to obtain the second timbre feature of the original audio signal, which indicates that the timbre of the same singer does not change over time.

[0076] Artist classification module 44: used to input the first timbre feature into the artist classification network, and output the artist classification result of the original audio signal through the artist classification network; specifically, before inputting the first timbre feature into the artist classification network, the vector quantization result and the second timbre feature are fused, and the fusion result is input into the decoder for spectrum reconstruction, and the first timbre feature is constrained to be a frame-by-frame timbre feature by using the spectrum reconstruction task; wherein, inputting the fusion result into the decoder for spectrum reconstruction includes: based on the fusion result of the vector quantization result and the second timbre feature, performing spectrum reconstruction by using the decoder to obtain a reconstructed second spectrum X'; completing the spectrum reconstruction task by calculating the MAE loss for the first spectrum X and the second spectrum X', and constraining the vector quantization result and the second timbre feature through the spectrum reconstruction task, so that the vector quantization result contains as much audio content information as possible, the second timbre feature contains as much timbre information as possible, and the first timbre feature is constrained to be a frame-by-frame timbre feature.

[0077] In the embodiment of the present application, the artist classification network adopts a CRNN structure, including 4 layers of 2D convolutional layers, 2 layers of GRU-RNN layers, and a final fully connected layer. First, the 2D convolutional layers are used to learn the spatial characteristics of the frame-by-frame timbre features. Then, the GRU-RNN network is used to learn the temporal characteristics of the frame-by-frame timbre features, enhancing the classification and recognition of the frame-by-frame timbre features in multiple dimensions. Finally, the fully connected layer is connected to the category for softmax classification and recognition, and the singer label corresponding to the original audio signal is output. By performing artist classification based on the frame-by-frame timbre features, the embodiment of the present application enhances the classification and recognition of the frame-by-frame timbre features in multiple dimensions and improves the accuracy of artist classification.

[0078] Based on the above, the artist classification system of the embodiment of the present invention represents audio content information by introducing a vector quantization method, and uses discrete codeword variable representation to reduce the data volume of continuous variables, thereby better reflecting the timbre information of the audio signal; by fusing the timbre information and audio content information and reconstructing the spectrum, the accurate decoupling of the audio content information and the timbre information is achieved, and the timbre information is constrained to frame-by-frame timbre features. Finally, artist classification is performed based on the frame-by-frame timbre features, thereby enhancing the classification and recognition of the frame-by-frame timbre features in multiple dimensions and improving the accuracy of artist classification.

[0079] See also Figure 4 , which is a schematic diagram of a terminal structure according to an embodiment of the present invention. The terminal 50 includes a processor 51 and a memory 52 coupled to the processor 51.

[0080] The memory 52 stores program instructions for implementing the above-mentioned artist classification method.

[0081] The processor 51 is configured to execute program instructions stored in the memory 52 to perform artist classification operations.

[0082] The processor 51 may also be referred to as a CPU (Central Processing Unit). The processor 51 may be an integrated circuit chip having signal processing capabilities. The processor 51 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor may be a microprocessor or any conventional processor.

[0083] See also Figure 5 , Figure 5 Schematic diagram of the structure of the storage medium of an embodiment of the present invention. The storage medium of the embodiment of the present invention stores a program file 61 that can implement all the above methods, wherein the program file 61 can be stored in the above storage medium in the form of a software product, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.

[0084] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0085] In addition, the functional units in the various embodiments of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software functional units. The above is only an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for classifying artists, characterized in that: include: Obtaining a first spectrum of the original audio signal, and encoding the first spectrum using an encoder to obtain an encoded continuous variable V, wherein the continuous variable V contains timbre information and audio content information of the original audio signal; Performing instance regularization on the continuous variable V, and performing vector quantization on the instance regularization result using discrete codewords in a codebook to obtain a vector quantization result, wherein the vector quantization result represents audio content information of the original audio signal; Subtracting the continuous variable from the vector quantization result to obtain a difference between the continuous variable and the vector quantization result, and using the difference as a first timbre feature of the original audio signal, where the first timbre feature is a frame-by-frame variation feature of the timbre in the original audio signal; Subtracting the continuous variable from the vector quantization result to obtain the difference between the continuous variable and the vector quantization result further includes: Performing frame-by-frame averaging on a difference result between the continuous variable and the vector quantization result, and replicating and expanding the result according to the number of frames to obtain a second timbre feature of the original audio signal, wherein the second timbre feature indicates that the timbre of the same singer does not change over time; fusing the vector quantization result with the second timbre feature, and inputting the fusion result into a decoder for spectrum reconstruction, wherein the spectrum reconstruction task is used to constrain the first timbre feature to be a frame-by-frame timbre feature; The first timbre feature is input into an artist classification network, and the artist classification result of the original audio signal is output through the artist classification network.

2. The artist classification method according to claim 1, characterized in that: The obtaining of the first spectrum of the original audio signal comprises: The original audio signal is input into the pre-trained vocal accompaniment separation model, and the vocal dry sound in the original audio signal is extracted by the vocal accompaniment separation model; Perform short-time Fourier transform on the extracted dry vocal sound to obtain a first spectrum X of the original audio signal.

3. The artist classification method according to claim 2, characterized in that: The performing instance regularization on the continuous variable V includes: For each continuous variable V, calculate its mean u and variance σ^; Normalize the continuous variable V according to the mean u and variance σ^ to obtain an instance regularization result V'; The normalization formula is: where ε is the bias term.

4. The artist classification method according to claim 3, characterized in that: The vector quantization of the instance regularization result using discrete codewords in the codebook includes: A codeword search is performed on the instance regularization result V', the distance between the instance regularization result V' and each codeword in the codebook is calculated, and argmin is calculated to obtain the codeword with the closest distance to represent the instance regularization result V'.

5. The artist classification method according to claim 1, characterized in that: Before inputting the first timbre feature into the artist classification network, the method further includes: Inputting the fusion result into the decoder for spectrum reconstruction includes: Based on the fusion result of the vector quantization result and the second timbre feature, a decoder is used to perform spectrum reconstruction to obtain a reconstructed second spectrum X'; A spectrum reconstruction task is completed by calculating the MAE loss for the first spectrum X and the second spectrum X'. The vector quantization result and the second timbre feature are constrained by the spectrum reconstruction task so that the vector quantization result contains as much audio content information as possible, the second timbre feature contains as much timbre information as possible, and the first timbre feature is constrained to be a frame-by-frame timbre feature.

6. The artist classification method according to claim 5, characterized in that: Inputting the first timbre feature into an artist classification network and outputting an artist classification result of the original audio signal through the artist classification network includes: The artist classification network includes a 2D convolutional layer, a GRU-RNN layer and a fully connected layer. The 2D convolutional layer is used to learn the features of the first timbre feature in the spatial dimension, and the GRU-RNN network is used to learn the features of the first timbre feature in the time domain. It is connected to the category through the fully connected layer for softmax classification and recognition, and outputs the singer label corresponding to the original audio signal.

7. An artist classification system, the system being used to implement the steps of the artist classification method according to any one of claims 1 to 6, characterized in that: include: Coding calculation module: used to obtain a first spectrum of the original audio signal and encode the first spectrum using an encoder to obtain an encoded continuous variable V, wherein the continuous variable V contains the timbre information and audio content information of the original audio signal; A vector quantization module is configured to perform instance regularization on the continuous variable V and perform vector quantization on the instance regularization result using discrete codewords in a codebook to obtain a vector quantization result, wherein the vector quantization result represents audio content information of the original audio signal; A timbre acquisition module is configured to perform a subtraction between the continuous variable and the vector quantization result to obtain a difference between the continuous variable and the vector quantization result, and use the difference as a first timbre feature of the original audio signal, wherein the first timbre feature is a frame-by-frame variation feature of the timbre in the original audio signal; Artist classification module: used to input the first timbre feature into the artist classification network, and output the artist classification result of the original audio signal through the artist classification network.

8. A terminal, characterized in that: The terminal includes a processor and a memory coupled to the processor, wherein: The memory stores program instructions for implementing the artist classification method according to any one of claims 1 to 6; The processor is configured to execute the program instructions stored in the memory to perform the artist classification method.

9. A storage medium, characterized in that: Program instructions executable by a processor are stored, and the program instructions are used to execute the artist classification method according to any one of claims 1 to 6.

Images