Song detection method and device, computer device, and storage medium

Abstract

The present application relates to the technical field of artificial intelligence, and discloses a song detection method and device, computer equipment and a storage medium. The method extracts dry sound from a to-be-processed audio file to obtain a dry audio signal after accompaniment removal. The extracted dry audio signal usually has a certain degree of noise. The method then extracts features from the dry audio signal to obtain dry sound features for subsequent song detection. A song recognition model is used to identify the dry sound features to obtain more accurate dry sound recognition results. The dry sound recognition results are filtered and de-noised to obtain final song detection results. The song recognition model is used to perform secondary detection on the dry sound features, thereby improving the accuracy of the song detection results.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a singing voice detection method and device, computer equipment and a storage medium. BACKGROUND

[0002] Singing voice detection is a key sub-task in music information retrieval tasks, and is usually used as a preprocessing step for other research tasks, such as singer identification tasks, which can identify singing voice segments. Singing voice detection is a task of marking the start and end positions of singing voice in a given audio through machine learning or signal processing. In existing singing voice detection technologies, relevant audio features such as MFCC and other spectral features are extracted after the audio is framed, and then a deep model is trained to finally complete the binary classification task of singing voice and non-singing voice.

[0003] In existing singing voice detection technologies, feature learning and classification networks are constructed through feature engineering selection and deep models, and rely on a large amount of supervised data, such as specific time labels in the data set, which are used to distinguish singing voice labels and non-singing voice labels. The labeled data is input into the network for supervised training. The above technology relies on a large amount of data, has high detection model training cost, and has the problem of insufficient accuracy of singing voice detection results. SUMMARY

[0004] The embodiments of the present application provide a singing voice detection method, device, computer equipment and storage medium, which solve the problem of insufficient accuracy of existing singing voice detection results.

[0005] The embodiments of the present application provide a singing voice detection method, comprising:

[0006] Obtaining a to-be-processed audio file;

[0007] Extracting dry sound from the to-be-processed audio file to obtain a dry sound audio signal;

[0008] Extracting features from the dry sound audio signal to obtain dry sound features;

[0009] Identifying the dry sound features using a singing voice recognition model to obtain a dry sound recognition result;

[0010] Filtering the dry sound recognition result to obtain a singing voice detection result.

[0011] The embodiments of the present application also provide a singing voice detection device, comprising:

[0012] A to-be-processed audio file acquisition module acquires a to-be-processed audio file;

[0013] The dry sound audio signal acquisition module extracts dry sound from the audio file to be processed to obtain a dry sound audio signal.

[0014] The dry sound feature acquisition module extracts features from the dry sound audio signal to obtain dry sound features.

[0015] The dry sound recognition result acquisition module identifies the dry sound features using a song voice recognition model to obtain a dry sound recognition result.

[0016] The song voice detection result acquisition module filters the dry sound recognition result to obtain a song voice detection result.

[0017] The embodiment of the present application also provides a computer device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the song voice detection method when executing the computer program.

[0018] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the song voice detection method.

[0019] The song voice detection method, device, computer device and storage medium described above extract dry sound from an audio file to be processed to obtain a dry sound audio signal after removing accompaniment, and the extracted dry sound audio signal usually has a certain degree of noise. The dry sound features extracted from the dry sound audio signal are used for subsequent song voice detection. The dry sound features are identified using a song voice recognition model to obtain more accurate dry sound recognition results. The dry sound recognition results are filtered to reduce noise and obtain final song voice detection results. The dry sound features are detected twice using the song voice recognition model, thereby improving the accuracy of the song voice detection results. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0021] Figure 1 is an application environment diagram of the song voice detection method in an embodiment of the present application;

[0022] Figure 2 is a flowchart of the song voice detection method in an embodiment of the present application;

[0023] Figure 3This is another flowchart of a singing detection method in one embodiment of the present invention;

[0024] Figure 4 This is another flowchart of a singing detection method in one embodiment of the present invention;

[0025] Figure 5 This is another flowchart of a singing detection method in one embodiment of the present invention;

[0026] Figure 6 This is another flowchart of a singing detection method in one embodiment of the present invention;

[0027] Figure 7 This is a schematic diagram of a singing detection device according to an embodiment of the present invention;

[0028] Figure 8 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] The singing voice detection method provided in this embodiment of the invention can be applied to, for example... Figure 1 The application environment shown. Figure 1 As shown, the client (computer device) communicates with the server via a network. The client, also known as the user terminal, refers to the program that provides local services to the client, corresponding to the server. Client (computer device) includes, but is not limited to, various personal computers, laptops, smartphones, tablets, cameras, and portable wearable devices. The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0031] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0032] The song detection method provided by the embodiment of the present application can be applied in the application environment as shown in the figure. Figure 1 Specifically, the song detection method is applied in a song detection system, and the song detection system comprises a client and a server as shown in the figure. Figure 1 The client and the server communicate through a network, and are used to obtain a corresponding song detection result for a to-be-processed audio file, so as to reduce the training cost of a song recognition model.

[0033] In an embodiment, as shown in the figure, a song detection method is provided. Figure 2 The server in the figure is taken as an example for illustration, and the method comprises the following steps. Figure 1

[0034] S201: obtaining a to-be-processed audio file;

[0035] S202: extracting dry sound from the to-be-processed audio file to obtain a dry sound audio signal;

[0036] S203: extracting features from the dry sound audio signal to obtain dry sound features;

[0037] S204: identifying the dry sound features by using a song recognition model to obtain a dry sound recognition result;

[0038] S205: filtering the dry sound recognition result to obtain a song detection result.

[0039] The dry sound is also called naked sound, which belongs to the audio terminology, and generally refers to the pure human voice after recording without any spatial property or post-processing and processing. In contrast, the human voice after spatial property post-processing (such as reverberation, delay, etc.) is called wet sound. Dry sound and wet sound generally refer to pure human voice without music itself. Dry sound is the original version, and wet sound is the post-processed version.

[0040] As an example, in step S201, the server receives a to-be-processed audio file provided by a user through a client, so as to perform song detection processing based on the to-be-processed audio file. In this example, the content of the to-be-processed audio file takes a song as an example, and the format of the to-be-processed audio file includes but is not limited to CDA format, WAV format, MP3 format, MP3 PRO format, APE format, FLAC format, AAC format, Real Media format and Windows Media format, etc.

[0041] ​As an example, in step S202, the server performs dry sound extraction processing on the obtained to-be-processed audio file. Generally, the to-be-processed audio file includes accompaniment and dry sound. The dry sound in the to-be-processed audio file is separated and extracted to obtain a dry sound audio signal in the to-be-processed audio file, which is used in subsequent singing voice detection processing to exclude accompaniment interference, thereby helping to improve detection efficiency and detection accuracy.

[0042] The audio signal refers to a carrier of frequency and amplitude change information of a sound wave, and is generally a series of digital identifiers representing continuous changes in the time domain. The dry sound audio signal refers to the audio signal corresponding to the pure vocal part of the dry sound.

[0043] As an example, in step S203, after obtaining the dry sound audio signal, the server performs audio feature extraction on the dry sound audio signal to obtain dry sound features recognizable by machine learning, and then performs subsequent singing voice detection processing on the extracted dry sound features.

[0044] The dry sound features extracted by the audio feature extraction method include but are not limited to features directly extracted from the audio signal (such as zero-crossing rate), features obtained by converting the signal to frequency (such as spectral centroid), features obtained by a specific model (such as melody), and features obtained by changing the quantization feature scale inspired by human auditory perception (such as MFCCs).

[0045] As an example, in step S204, after obtaining the dry sound features extracted by the server, the server performs recognition processing on the dry sound features using a singing voice recognition model to obtain a singing voice recognition result. In this example, the main architecture of the singing voice recognition model is a 4-layer convolutional network and a 2-layer LSTM network. The singing voice recognition model can be used for dry sound recognition. It should be noted that the singing voice recognition model does not need to be pre-trained, and when processing the separated dry sound audio signal, it can also achieve the effect of blind detection. The singing voice detection result can be used for downstream tasks such as singer recognition tasks and song recognition tasks.

[0046] As an example, in step S205, the server performs filtering processing on the obtained dry sound recognition result to complete the de-noising processing, thereby solving the dry sound features with noise points and making the singing voice detection result more robust, which helps to ensure the accuracy of the singing voice detection result.

[0047] The filtering is an operation of filtering out specific waveband frequencies in the signal, and is generally used for de-noising processing, which is an important measure to suppress and prevent interference.

[0048] In the example, the dry sound of the audio file to be processed is extracted to obtain the dry audio signal after the accompaniment is removed, and the dry audio signal after extraction usually has a certain degree of noise. Then, the dry audio signal is subjected to feature extraction, the dry sound feature is recognized by using the song voice recognition model, a more accurate dry sound recognition result is obtained, the dry sound recognition result is filtered and de-noised to obtain the final song voice detection result, and the dry sound feature is detected again by using the song voice recognition model, so that the accuracy of the song voice detection result is improved.

[0049] In an embodiment, as shown in Figure 3 Step S202, dry sound extraction is performed on the audio file to be processed to obtain a dry audio signal, including:

[0050] S301: using a target U-Net model, converting and processing the audio file to be processed to obtain an audio spectrum to be processed;

[0051] S302: separating the accompaniment from the audio file to be processed according to the audio spectrum to be processed to obtain a dry audio signal.

[0052] The spectrum is the abbreviation of frequency spectrum density, which is the distribution curve of frequency. Complex oscillation is decomposed into harmonic oscillation with different amplitudes and frequencies. The graph of the amplitude of these harmonic oscillations arranged by frequency is called frequency spectrum, and the frequency spectrum decomposed from sound vibration is called audio spectrum or sound spectrum.

[0053] As an example, in step S301, after the server obtains the audio file to be processed uploaded by the user, the target U-Net model is used to convert and process the audio file to be processed to obtain an audio spectrum to be processed with dry sound and accompaniment. In the example, the network architecture of the target U-Net model is mainly composed of 7 layers of convolution and 7 layers of deconvolution network. The skip link is added between the layers with the same data size to form the convolution layer flow of the bottom layer data to the high layer with the same resolution. According to the U-Net model parameters after training, the corresponding target U-Net model is obtained after updating, which is used to extract the dry sound part of the audio signal in the audio file.

[0054] The U-Net is a deep learning network based on the full convolution network (FCN). Because the network structure is roughly in the form of U, it is called U-Net network. The difference between the convolutional neural network (CNN) and the full convolutional neural network (FCN) is that the FCN replaces the last fully connected layer (FC) of the CNN with a convolution layer (Conv).

[0055] As an example, in step S302, the server identifies the accompaniment audio spectrum and the dry audio spectrum in the target U-Net model for the converted to-be-processed audio spectrum, and separates the accompaniment audio spectrum and the dry audio spectrum to obtain the accompaniment audio signal corresponding to the accompaniment audio spectrum and the dry audio signal corresponding to the dry audio spectrum. The dry audio signal is reserved for subsequent singing detection processing.

[0056] In the present example, the accompaniment and dry sound of the to-be-processed audio file are separated by the trained target U-Net model to preliminarily obtain the corresponding dry audio signal, thereby improving the acquisition efficiency of the dry audio signal and achieving blind detection when the separated dry audio signal is processed subsequently.

[0057] In an embodiment, as shown in Figure 4 Step S203, the dry audio signal is subjected to feature extraction to obtain dry sound features, including:

[0058] S401: Pre-emphasizing the dry audio signal to obtain an emphasized dry sound signal;

[0059] S402: Framing and windowing the emphasized dry sound signal to obtain a time-domain dry sound signal;

[0060] S403: Processing the time-domain dry sound signal to obtain a time-domain dry sound spectrum;

[0061] S404: Extracting features from the time-domain dry sound spectrum to obtain dry sound features.

[0062] In the present example, the Mel-Frequency Cepstral Coefficients (MFCCs) corresponding to the dry audio signal are extracted for subsequent singing detection processing.

[0063] Mel-Frequency Cepstral Coefficients (MFCCs) are coefficients that make up the Mel-Frequency Cepstrum. The Mel-Frequency Cepstrum is a linear transformation of the log power spectrum based on a nonlinear mel scale of frequency. The Mel-Frequency Cepstrum is based on the principle of the cepstrum of an audio segment. The difference between the cepstrum and the Mel-Frequency Cepstrum is that the frequency band division of the Mel-Frequency Cepstrum is equally divided on the mel scale, which is more approximate to the human auditory system than the linearly spaced frequency bands used in normal log cepstrum.

[0064] As an example, in step S401, the server performs pre-emphasis processing on the obtained dry sound signal to obtain an emphasized dry sound signal. In this example, the purpose of pre-emphasis is to filter out low-frequency components in the data, making the high-frequency characteristics more prominent, thereby enhancing the high-frequency part of the dry sound signal, making the signal spectrum flat, maintaining the entire frequency band from low to high frequency, and being able to obtain the spectrum with the same signal-to-noise ratio. At the same time, it is also to eliminate the effects of the vocal cords and lips during the process, to compensate for the high-frequency part of the speech signal suppressed by the pronunciation system.

[0065] As an example, in step S402, the server performs frame division and windowing processing on the obtained emphasized dry sound signal to obtain a time-domain dry sound signal with a corresponding frame length identifier. In this example, frame division is to group the sampling points corresponding to N time periods in the emphasized dry sound signal into an observation unit. In order to avoid too large changes between adjacent two frames, there is an overlapping area between the two adjacent frames. Windowing is to add a Hamming window to the dry sound signal corresponding to each frame length identifier to increase the continuity of the left and right ends of the dry sound signal corresponding to the frame length identifier, to prevent the converted spectrum from being very different from the original signal spectrum.

[0066] As an example, in step S403, the server performs time-domain analysis processing on the obtained time-domain dry sound signal to obtain a time-domain dry sound spectrum with time-varying characteristics, solving the problem that the ordinary spectrum only shows the frequency distribution of the signal, but does not have time-domain information, and cannot see the change of the frequency distribution with time.

[0067] Among them, the time-domain analysis method includes but is not limited to short-time Fourier, wavelet and Wigner distribution and other methods. In this example, the short-time Fourier transform method is used to perform Fourier transform processing on the short-time frame length identifier corresponding to the time-domain dry sound signal segment according to the frame length identifier of the time-domain dry sound signal, to obtain a time-domain dry sound spectrum with time-varying characteristics.

[0068] As an example, in step S404, after obtaining the time-domain dry sound spectrum, the server performs feature extraction of the Mel frequency cepstral coefficient to obtain the Mel frequency cepstral coefficient as the corresponding dry sound feature.

[0069] In this example, by performing frame division, Hamming windowing, and then short-time Fourier transform on the obtained dry sound signal to obtain the corresponding time-domain dry sound spectrum, and then performing feature extraction on the time-domain dry sound spectrum to obtain the Mel frequency cepstral coefficient as the corresponding dry sound feature, the use of the Mel frequency cepstral coefficient can improve the quality of dry sound data processing.

[0070] In an embodiment, in step S404, the dry sound signal is subjected to feature extraction to obtain a dry sound feature, including:

[0071] S4041: Obtain a dry sound energy spectrum based on the time-domain dry sound spectrum;

[0072] S4042: Perform Mel filtering on the dry sound energy spectrum to obtain a log power spectrum;

[0073] S4043: Process the log power spectrum to obtain a dry sound feature.

[0074] As an example, in step S4041, the server obtains a dry sound energy spectrum corresponding to the time-domain dry sound spectrum by performing square calculation on the obtained time-domain dry sound spectrum. The energy is a value represented by sound on a frequency spectrum, and the dry sound energy is a value represented by the extracted dry sound on the frequency spectrum, and thus the dry sound energy spectrum is a spectrum showing the dry sound energy.

[0075] As an example, in step S4042, the server performs Mel filtering on the obtained dry sound energy spectrum by using a filter, superimposes the corresponding frequency band dry sound energy on each filter frequency band, and finally performs logarithmic operation on the power spectrum output by each filter to obtain a log power spectrum corresponding to the dry sound energy spectrum. The filter frequency is set according to actual business requirements, that is, it corresponds to the frame length identifier.

[0076] As an example, in step S4043, the server obtains a mel frequency cepstral coefficient by performing calculation on the obtained log power spectrum by using inverse discrete cosine, and obtains a dry sound feature by performing first-order difference processing on the mel frequency cepstral coefficient.

[0077] In this example, the mel frequency cepstral coefficient is obtained by performing processing on the obtained time-domain dry sound spectrum and performing Mel filtering on the processed dry sound energy spectrum, and the dry sound feature with more significant characteristics is obtained by performing first-order difference processing.

[0078] In an embodiment, as shown in FIG. 2, Figure 5 In step S204, a song voice recognition model is used to perform recognition processing on the dry sound feature to obtain a dry sound recognition result, including:

[0079] S501: Process the dry sound feature according to the frame length identifier corresponding to the dry sound feature to obtain a dry sound time sequence feature;

[0080] S502: Perform dimension reduction processing on the dry sound time sequence feature to obtain a dry sound comprehensive feature;

[0081] S503: Perform recognition processing on the dry sound comprehensive feature corresponding to each frame length identifier to obtain a dry sound recognition result.

[0082] As an example, in step S501, after obtaining the dry sound features, the server performs association processing on the dry sound features and the frame length identifiers corresponding to the dry sound features by using the song voice recognition model, to perform deep feature extraction processing on the dry sound features, to obtain dry sound time sequence features with more significant features and time attributes. In this example, the frame length identifiers corresponding to the dry sound features are taken as the horizontal axis, and the dry sound features are taken as the vertical axis, to obtain two-dimensional dry sound time sequence features.

[0083] The main architecture of the song voice recognition model adopts a 4-layer convolutional network and a 2-layer LSTM network, the convolutional network is used to process the dry sound features and obtain corresponding dry sound time sequence features, and the LSTM network is used to process the dry sound time sequence features to perform dry sound recognition.

[0084] As an example, in step S502, after obtaining the dry sound time sequence features, the server performs dimension reduction processing on the two-dimensional dry sound time sequence features to obtain one-dimensional dry sound comprehensive features. In this example, the server can perform mean value processing on the two-dimensional dry sound time sequence features to obtain one-dimensional dry sound comprehensive features. The dry sound comprehensive features are one-dimensional integrated features, which are obtained by performing mean value processing on the dry sound features corresponding to each frame length identifier.

[0085] As an example, in step S503, after obtaining the dry sound comprehensive features, the server performs dry sound recognition processing on the dry sound comprehensive features according to the dry sound comprehensive features of each frame length identifier to obtain dry sound recognition results. In this example, the dry sound comprehensive features are output to a sigmoid node of a neuron to determine whether they are dry sound or non-dry sound, and the dry sound recognition results are obtained.

[0086] In this example, the song voice recognition model is used to process the dry sound features according to the frame length identifiers corresponding to the dry sound features, to obtain dry sound time sequence features with time attributes, and the mean value processing is performed on the dry sound time sequence features to obtain dry sound comprehensive features for dry sound recognition, to obtain the final dry sound recognition results, thereby improving the recognition degree of dry sound in the audio signal.

[0087] In an embodiment, as shown in FIG. 2, Figure 6 In step S205, the dry sound recognition results are filtered to obtain song voice detection results, including:

[0088] S601: performing smoothing processing on the dry sound recognition results according to the frame length identifiers corresponding to the dry sound features, to obtain smoothing processing results of the dry sound features corresponding to each frame length identifier;

[0089] S602: obtaining the song voice detection results based on the smoothing processing results.

[0090] As an example, in step S601, after confirming the dry sound recognition result, the server smoothes the dry sound recognition result corresponding to each frame length identifier to perform de-noising processing, and obtains the smoothing processing result of the dry sound feature corresponding to each frame length identifier after de-noising. In this example, time domain smoothing filtering is used to perform frame-by-frame smoothing processing on the frame-by-frame detected dry sound recognition result to obtain at least one smoothing processing result corresponding to each frame length identifier.

[0091] As an example, in step S602, the server integrates the obtained at least one smoothing processing result to obtain a final singing voice detection result. In this example, according to the frame length identifier, the corresponding frame-by-frame smoothing processing is performed, the smoothing processing result corresponding to at least one frame length identifier is integrated, and the final singing voice detection result is obtained. The median value of the dry sound recognition result data corresponding to the frame length identifier is calculated as the smoothing processing result corresponding to each frame length identifier, and then the dry sound recognition result corresponding to the frame length identifier is continuously slid to perform smoothing processing until the entire output result is processed once to obtain the final singing voice detection result.

[0092] In this example, according to the frame length identifier corresponding to the dry sound feature, the dry sound recognition result is smoothed, the smoothing processing result of the dry sound feature corresponding to each frame length identifier is integrated, and the singing voice detection result is obtained, so that the singing voice detection result is obtained by using the smoothing filtering processing, and the robustness of the detection result of the abnormal data frame of the noise point is improved.

[0093] In another embodiment, before step S201 of obtaining the audio file to be processed, the singing voice detection method further comprises:

[0094] S701: Obtain a training accompaniment spectrum and a training dry sound spectrum;

[0095] S702: Input the training accompaniment spectrum and the training dry sound spectrum into a U-Net model for training to obtain a target U-Net model.

[0096] As an example, in step S701, the server obtains a training accompaniment spectrum and a training dry sound spectrum for training the U-Net model, so that the U-Net model can more accurately recognize the accompaniment audio signal and the dry sound audio signal, thereby performing preliminary dry sound audio signal extraction in this example, reducing noise points as much as possible, and improving the accuracy of dry sound extraction.

[0097] As an example, in step S702, the server inputs the obtained training accompaniment spectrum and training dry sound spectrum into the U-Net model, increases the skip link between layers with the same data size, forms the convolution layer flow of the bottom layer data to the high layer with the same resolution, and updates the corresponding target U-Net model according to the U-Net model parameters after training.

[0098] In the present example, by inputting the training accompaniment spectrum and the training dry sound spectrum into the U-Net model, the U-Net model parameters are updated, and a trained target U-Net model is obtained, which is used for separating the accompaniment and dry sound of the to-be-processed audio file, improving the efficiency of obtaining the dry sound audio signal, and at the same time, when the separated dry sound audio signal is processed subsequently in the present example, the effect of blind detection is achieved.

[0099] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0100] In an embodiment, a singing voice detection device is provided, which corresponds to the singing voice detection method in the above embodiment. As shown in the figure, the singing voice detection device includes a to-be-processed audio file acquisition module 801, a dry sound audio signal acquisition module 802, a dry sound feature acquisition module 803, a dry sound recognition result acquisition module 804, and a singing voice detection result acquisition module 805. The functions of each module are described in detail as follows: Figure 7

[0101] The to-be-processed audio file acquisition module 801 acquires the to-be-processed audio file;

[0102] The dry sound audio signal acquisition module 802 extracts dry sound from the to-be-processed audio file to obtain a dry sound audio signal;

[0103] The dry sound feature acquisition module 803 extracts features from the dry sound audio signal to obtain dry sound features;

[0104] The dry sound recognition result acquisition module 804 identifies the dry sound features using a singing voice recognition model to obtain a dry sound recognition result;

[0105] The singing voice detection result acquisition module 805 filters the dry sound recognition result to obtain a singing voice detection result.

[0106] In an embodiment, the dry sound audio signal acquisition module 802 includes:

[0107] The to-be-processed audio spectrum acquisition unit converts the to-be-processed audio file using a target U-Net model to obtain a to-be-processed audio spectrum;

[0108] The dry sound audio signal acquisition unit separates the accompaniment from the to-be-processed audio file based on the to-be-processed audio spectrum to obtain a dry sound audio signal.

[0109] In an embodiment, the dry sound feature acquisition module 803 includes:

[0110] ​The dry sound signal obtaining unit obtains a dry sound signal.

[0111] The time-domain dry sound signal obtaining unit obtains a time-domain dry sound signal by framing and windowing the emphasized dry sound signal.

[0112] The time-domain dry sound spectrum obtaining unit obtains a time-domain dry sound spectrum by processing the time-domain dry sound signal.

[0113] The dry sound feature obtaining unit obtains a dry sound feature by extracting a feature from the time-domain dry sound spectrum.

[0114] In an embodiment, the dry sound feature obtaining unit comprises:

[0115] The dry sound energy spectrum obtaining sub-unit obtains a dry sound energy spectrum based on the time-domain dry sound spectrum.

[0116] The log power spectrum obtaining sub-unit obtains a log power spectrum by performing a Mel filter on the dry sound energy spectrum.

[0117] The dry sound feature obtaining sub-unit obtains a dry sound feature by processing the log power spectrum.

[0118] In an embodiment, the dry sound recognition result obtaining module 804 comprises:

[0119] The dry sound time sequence feature obtaining unit obtains a dry sound time sequence feature by processing the dry sound feature according to the frame length identifier corresponding to the dry sound feature.

[0120] The dry sound comprehensive feature obtaining unit obtains a dry sound comprehensive feature by performing dimension reduction processing on the dry sound time sequence feature.

[0121] The dry sound recognition result obtaining unit obtains a dry sound recognition result by performing recognition processing on the dry sound comprehensive feature corresponding to each frame length identifier.

[0122] In an embodiment, the singing voice detection result obtaining module 805 comprises:

[0123] The smoothing processing result obtaining unit obtains a smoothing processing result of the dry sound feature corresponding to each frame length identifier by performing smoothing processing on the dry sound recognition result according to the frame length identifier corresponding to the dry sound feature.

[0124] The singing voice detection result obtaining unit obtains a singing voice detection result based on the smoothing processing results.

[0125] In another embodiment, the singing voice detection method further comprises:

[0126] The training spectrum obtaining module obtains a training accompaniment spectrum and a training dry sound spectrum.

[0127] The target U-Net model acquisition module inputs the training accompaniment spectrum and the training dry sound spectrum into the U-Net model for training to acquire the target U-Net model.

[0128] The specific limitations of the song detection device can refer to the limitations of the song detection method in the foregoing, which will not be repeated here. Each module in the above song detection device can be realized by software, hardware and a combination thereof in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so that the processor calls and executes the operations corresponding to each of the above modules.

[0129] In one embodiment, a computer device is provided, which can be a server, and an internal structure diagram thereof can be as shown in Figure 8 The computer device includes a processor, a memory, a network interface and a database connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data used or generated in the process of the song detection method. The network interface of the computer device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement a song detection method.

[0130] In an embodiment, a computer device is provided, which includes a memory, a processor and a computer program stored in the memory and executable on the processor. The processor implements the song detection method in the above embodiments when executing the computer program, such as Figure 2 shown in S201-S205, or Figures 3 to 6 For brevity, the details will not be repeated here. Alternatively, the processor implements the functions of each module / unit in this embodiment of the song detection device when executing the computer program, such as Figure 7 the functions of the to-be-processed audio file acquisition module 801, the dry sound audio signal acquisition module 802, the dry sound feature acquisition module 803, the dry sound recognition result acquisition module 804 and the song detection result acquisition module 805 shown in

[0131] In an embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by the processor to implement the song detection method in the above embodiments, such as Figure 2 shown in S201-S205, or Figures 3 to 6As shown in the computer program product 1000, to avoid repetition, the functions of the song detection device in the above embodiments are not repeated here. Alternatively, the computer program is executed by the processor to implement the functions of the modules / units in the above embodiments of the song detection device, for example Figure 7 As shown in the computer program product 1000, to avoid repetition, the functions of the song detection device in the above embodiments are not repeated here. Alternatively, the computer program is executed by the processor to implement the functions of the modules / units in the above embodiments of the song detection device, for example

[0132] Those skilled in the art can understand that all or part of the processes in the above embodiments can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above embodiments. Any reference to memory, storage, database or other medium in the embodiments of the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0133] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of functional units and modules is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the above-described functions.

[0134] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit it; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for detecting singing voice, characterized in that, include: Obtain the audio file to be processed; The target U-Net model is used to convert the audio file to be processed to obtain the audio spectrum of the audio to be processed, which has dry sound and accompaniment; and the dry sound audio spectrum is separated from the audio spectrum of the audio to be processed, and the dry sound audio signal corresponding to the dry sound audio spectrum is retained. Feature extraction is performed on the dry audio signal to obtain dry audio features; By using the frame length identifier corresponding to the dry sound feature as the horizontal axis and the dry sound feature as the vertical axis, a singing recognition model is used to perform deep feature extraction on the dry sound feature to obtain two-dimensional dry sound temporal features. The dry sound temporal features are then dimensionality-reduced to obtain one-dimensional dry sound comprehensive features. The dry sound comprehensive features corresponding to each frame length identifier are then recognized to obtain dry sound recognition results. The dry sound recognition results are used to indicate whether the dry sound audio signal segment corresponding to the frame length identifier is dry sound or non-dry sound. Based on the frame length identifier corresponding to the dry sound feature, the dry sound recognition result is smoothed to obtain the smoothed result of the dry sound feature corresponding to each frame length identifier. The smoothing results of the dry sound features corresponding to each frame length identifier are integrated to obtain the singing sound detection result.

2. The singing voice detection method as described in claim 1, characterized in that, The step of extracting features from the dry audio signal to obtain dry audio features includes: The dry audio signal is pre-emphasized to obtain an emphasized dry audio signal; The stressed dry sound signal is framed and windowed to obtain the time-domain dry sound signal; The time-domain dry sound signal is processed to obtain the time-domain dry sound spectrum; The dry sound spectrum in the time domain is subjected to feature extraction to obtain dry sound features.

3. The singing voice detection method as described in claim 2, characterized in that, The step of extracting features from the time-domain dry sound spectrum to obtain dry sound features includes: Based on the aforementioned time-domain dry sound spectrum, the dry sound energy spectrum is obtained; Mel filtering is performed on the dry acoustic energy spectrum to obtain the logarithmic power spectrum; The logarithmic power spectrum is processed to obtain dry sound characteristics.

4. The singing voice detection method as described in claim 1, characterized in that, Before acquiring the audio file to be processed, the singing detection method further includes: Obtain the training accompaniment spectrum and the training dry audio spectrum; The training accompaniment spectrum and the training dry audio spectrum are input into the U-Net model for training to obtain the target U-Net model.

5. A singing voice detection device, characterized in that, include: The module for acquiring audio files to be processed retrieves the audio files to be processed. The dry audio signal acquisition module uses the target U-Net model to convert the audio file to be processed, and obtains the audio spectrum to be processed with dry sound and accompaniment; and separates the dry audio spectrum from the audio spectrum to be processed, and retains the dry audio signal corresponding to the dry audio spectrum. The dry sound feature acquisition module extracts features from the dry sound audio signal to acquire dry sound features. The dry sound recognition result acquisition module uses the frame length identifier corresponding to the dry sound feature as the horizontal axis and the dry sound feature as the vertical axis. It uses a singing recognition model to perform deep feature extraction processing on the dry sound feature to obtain two-dimensional dry sound temporal features. It then performs dimensionality reduction processing on the dry sound temporal features to obtain one-dimensional dry sound comprehensive features. Finally, it performs recognition processing on the dry sound comprehensive features corresponding to each frame length identifier to obtain the dry sound recognition result. The dry sound recognition result is used to indicate whether the dry sound audio signal segment corresponding to the frame length identifier is dry sound or non-dry sound. The singing detection result acquisition module smooths the dry sound recognition result according to the frame length identifier corresponding to the dry sound feature to obtain the smoothed result of the dry sound feature corresponding to each frame length identifier; and integrates the smoothed result of the dry sound feature corresponding to each frame length identifier to obtain the singing detection result.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the singing detection method as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the singing detection method as described in any one of claims 1 to 4.

Images