Audio classification method and device based on frequency spectrum transformation, equipment and storage medium

Abstract

The invention provides an audio classification method and device based on spectrum transformation, equipment and a storage medium, and relates to the technical field of audio processing. According to the method, the to-be-classified audio data is converted into the target spectrogram through the signal conversion algorithm, the time sequence information of the audio signals is converted into the frequency domain information, the storage requirement of the audio data is reduced, the continuous waveform data is converted into the image data, and the audio signals can be conveniently processed through the image processing technology. The pre-trained image classification model is used to classify and identify the target spectrogram, and the accuracy and efficiency of audio classification are improved. The audio processing is converted into the image processing, so that the storage efficiency is improved, the requirement of the storage space is reduced, and the audio classification efficiency and accuracy are improved by virtue of the maturity and the high efficiency of the image recognition technology.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to an audio classification method, device, equipment and storage medium based on spectrum transformation. Background Art

[0002] There are four commonly used methods of speech recognition technology: 1. Methods based on linguistics and acoustics, 2. Random model methods, 3. Methods using artificial neural networks, and 4. Probabilistic grammatical analysis.

[0003] Among them, the most mainstream method is the random model method. This method mainly uses the steps of extracting features, training templates, classifying templates, and judging templates to recognize speech. After the simulated speech signal is sampled to obtain waveform data, it must first be input into the feature extraction module to extract appropriate acoustic feature parameters for subsequent acoustic model training. However, the storage of waveform data consumes a lot of storage space, and the waveform data extraction and recognition technology is not as mature as the image extraction and recognition technology, which makes the efficiency of audio classification and recognition low.

[0004] Therefore, how to improve the efficiency of audio classification has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present application provides an audio classification method, apparatus, device and storage medium based on spectrum transformation, aiming to improve the efficiency of audio classification.

[0006] In a first aspect, the present application provides an audio classification method based on spectrum transformation, and the audio classification method based on spectrum transformation comprises the following steps:

[0007] Obtain audio data to be classified;

[0008] Based on a signal conversion algorithm, the audio data to be classified is converted into a signal to generate a target spectrum diagram;

[0009] Based on the pre-trained image classification model, the target spectrogram is classified and identified, and the audio classification result of the target spectrogram is output.

[0010] In a second aspect, the present application further provides an audio classification device based on spectrum transformation, the audio classification device based on spectrum transformation comprising:

[0011] A data acquisition module, used to acquire audio data to be classified;

[0012] A spectrum conversion module, used for performing signal conversion on the audio data to be classified based on a signal conversion algorithm to generate a target spectrum graph;

[0013] The model prediction module is used to classify and identify the target spectrogram based on a pre-trained image classification model, and output the audio classification result of the target spectrogram.

[0014] In a third aspect, the present application also provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the audio classification method based on spectrum transformation as described above are implemented.

[0015] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the above-mentioned audio classification method based on spectrum transformation are implemented.

[0016] The present application provides an audio classification method, device, computer equipment and storage medium based on spectrum transformation. The method of the present application includes obtaining audio data to be classified; based on a signal transformation algorithm, performing signal conversion on the audio data to be classified to generate a target spectrum graph; based on a pre-trained image classification model, classifying and identifying the target spectrum graph, and outputting the audio classification result of the target spectrum graph. In the above manner, the present application converts the audio data to be classified into a target spectrum graph through a signal transformation algorithm, and converts the timing information of the audio signal into frequency domain information, which not only reduces the storage requirements of the audio data, but also converts the continuous waveform data into image data, which is convenient for processing the audio signal through image processing technology. The target spectrum graph is classified and identified using a pre-trained image classification model, which improves the accuracy and efficiency of audio classification. By converting audio processing into image processing, not only the storage efficiency is improved and the storage space requirements are reduced, but also the efficiency and accuracy of audio classification are improved by virtue of the maturity and high efficiency of image recognition technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A schematic diagram of a flow chart of a first embodiment of an audio classification method based on spectrum transformation provided by the present application;

[0019] Figure 2 A schematic diagram of a flow chart of a second embodiment of an audio classification method based on spectrum transformation provided by the present application;

[0020] Figure 3 It is a structural schematic diagram of a first embodiment of an audio classification device based on spectrum transformation provided by the present application;

[0021] Figure 4 It is a schematic block diagram of the structure of a computer device provided in an embodiment of the present application.

[0022] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0025] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0026] Please refer to Figure 1 , Figure 1 A schematic diagram of a flow chart of a first embodiment of an audio classification method based on spectrum transformation provided by the present application.

[0027] like Figure 1 As shown, the audio classification method based on spectrum transformation includes steps S101 to S103.

[0028] S101, obtaining audio data to be classified;

[0029] In one embodiment, the audio data to be classified may be obtained from a variety of data sources, such as recordings, databases, or pre-recorded audio clips.

[0030] In one embodiment, the audio data to be classified may be audio data of the same data format or data type collected from the same data source, or may be multiple audio segment data collected from multiple data sources.

[0031] In one embodiment, the collected audio data to be classified may exist in different formats, such as WAV, MP3, AAC, etc. Therefore, after obtaining the audio data to be classified, the audio data to be classified can be preprocessed first to convert the audio data to be classified into audio data of a unified format.

[0032] Furthermore, based on the sampling rate of the audio data to be classified, the audio data to be classified is converted into a continuous digital sequence, so as to generate the target spectrogram according to the digitally represented audio data to be classified.

[0033] Generally speaking, sampling rate refers to the number of times an analog signal is sampled per second, usually expressed in Hertz (Hz). For example, the sampling rate of CD quality sound is 44.1kHz, which means 44,100 samples per second.

[0034] In one embodiment, sample rate conversion (Sample Rate Conversion) is also called resampling, which is a common operation in digital signal processing. During the sampling process, the continuous waveform of the analog signal is converted into a series of discrete sample values, which constitute a digital sequence.

[0035] In one embodiment, sampling rate conversion is to convert an audio signal from its original sampling rate to another sampling rate to meet different playback or processing requirements. For example, the standard sampling rate of CD audio is 44100Hz, while some professional audio processing systems may require a higher sampling rate, such as 96000Hz. During the conversion process, a low-pass filter is usually required to avoid aliasing, a process called decimation and interpolation.

[0036] Generally, various tools and libraries can be used to process audio data. For example, Python's scipy.signal module provides a resample function to convert the sampling rate.

[0037] For example, after sampling, each sample of the analog signal needs to be quantized, that is, the numerical representation of its amplitude is determined. Quantization usually involves the sampling depth, such as 16 bits or 24 bits, which determines the dynamic range of the signal and the possible sound quality. Assuming a sampling rate of 44.1kHz is selected, this means that we will collect 44,100 sample points per second. Using an analog-to-digital converter (ADC), we convert the analog signal value of each sample point into a digital value. For example, if the sampling depth is 16 bits, then each sample will be quantized into a 16-bit number ranging from -32768 to 32767. After sampling and quantization, a continuous sequence of numbers is obtained, for example: [-10000, -9500, -9000, ..., 0, ..., 9000, 9500, 10000], which represents the digital representation of the original analog signal and can be used to generate spectrograms for audio analysis and classification.

[0038] S102, performing signal conversion on the audio data to be classified based on a signal conversion algorithm to generate a target spectrum graph;

[0039] Generally, a spectrogram can show the distribution of audio signals at different frequencies, which is very useful for audio analysis and classification. Types of spectrograms include linear amplitude spectrum, logarithmic amplitude spectrum, and autopower spectrum. The logarithmic amplitude spectrum is often used in audio analysis because it can highlight low-amplitude signals.

[0040] In one embodiment, in order to generate a target spectrogram, it is necessary to perform a Fast Fourier Transform (FFT) on the digitally represented audio data, that is, convert the audio data to be classified from a time domain signal to a frequency domain signal.

[0041] It is understandable that audio data exists in the form of a time series, that is, a series of amplitude values ​​that change over time, and the frequency components of the signal cannot be directly observed in the time domain. However, performing FFT on the digitally represented audio data and converting the time domain signal into the frequency domain signal can convert the signal from the time domain to the frequency domain and reveal the frequency components of the signal.

[0042] Furthermore, based on the signal conversion algorithm, the audio data to be classified is converted into a frequency domain to obtain frequency information; based on the spectrum conversion algorithm, the frequency information is subjected to a spectrum conversion to generate the target spectrum graph.

[0043] In one embodiment, the signal transformation algorithm may be a fast Fourier transform (FFT), a short-time Fourier transform (STFT), or other similar frequency domain transformation methods.

[0044] Among them, FFT is an algorithm for efficiently calculating DFT (Discrete Fourier Transform), which can convert the time domain representation of a signal into its frequency domain representation, revealing the frequency components of the signal. STFT obtains the time-frequency representation of the signal by dividing the signal into a series of shorter time windows and applying FFT on each window. The results of STFT are usually displayed in the form of a spectrogram, that is, a graph of frequency changes over time.

[0045] Exemplarily, the FFT algorithm is used to convert the audio signal from the time domain to the frequency domain to obtain frequency information. The frequency domain information is converted into polar coordinates to extract amplitude information. The amplitude information is converted into decibel units to simulate the logarithmic perception of the human ear to sound. The frequency domain data after FFT processing is used to generate spectrograms, which will be used as input to the image classifier.

[0046] Specifically, the types of spectrum diagrams include linear amplitude spectrum, logarithmic amplitude spectrum and autopower spectrum, etc. The amplitude of each spectrum line in the logarithmic amplitude spectrum is calculated logarithmically, so the unit of its ordinate is dB (decibel). The purpose of this transformation is to make those components with lower amplitudes higher relative to the high amplitude components, so as to observe the periodic signal hidden in the low-amplitude noise.

[0047] In one embodiment, based on an image preprocessing algorithm, data processing is performed on the target spectrogram so that the target spectrogram meets the input parameter requirements of the image classification model.

[0048] In one embodiment, the generated spectrogram may be subjected to image preprocessing by an image preprocessing algorithm, which may include operations such as cropping, scaling, and normalization, to meet the input requirements of the classifier.

[0049] In one embodiment, the spectrum diagram may be converted into a grayscale image according to actual application requirements, or, if color channel information is used, it may be retained as a color image.

[0050] In one embodiment, data enhancement techniques such as rotation, flipping, cropping, etc. may also be applied to increase the diversity of the data set and improve the generalization ability of the model.

[0051] In one embodiment, conventional image processing methods may be used to extract features of the spectrogram, such as edge detection, texture analysis, etc. Alternatively, an image classification model may be directly used to automatically learn the feature representation of the spectrogram.

[0052] In one embodiment, feature extraction of audio signals is a key step in audio classification. Common features include Mel-frequency cepstral coefficients (MFCCs) and Mel-spectrograms. These features can capture key information in audio signals and help model classification.

[0053] S103: classify and identify the target spectrogram based on a pre-trained image classification model, and output an audio classification result of the target spectrogram.

[0054] In one embodiment, during the audio rapid classification and recognition process, the audio classification result generally refers to the prediction of the category to which the audio file belongs after the model analyzes the input spectrogram.

[0055] Specifically, the preprocessed and enhanced spectrogram is input into the trained graph classifier, and the image classification model extracts features from the input spectrogram through a multi-layer convolutional neural network (CNN). Each convolutional layer learns different levels of features of the spectrogram, from shallow edge and texture information to deep high-level abstract features. After feature extraction, the model maps these features to a probability distribution, which represents the possibility that the input spectrogram belongs to each predefined audio category. Based on the output probability distribution, the image classification model selects the category with the highest probability as the predicted category. Finally, the classification result output by the model is the predicted category of the audio file. For example, if the model predicts that the audio most likely belongs to the "classical music" category, then "classical music" is the classification result.

[0056] In one embodiment, a threshold can be set to determine the certainty of the classification according to the audio classification application requirements. The classification is considered reliable only when the prediction probability exceeds this threshold. The prediction results of the model can be verified for accuracy by manual review or comparison with data with known labels. The performance of the classification results is evaluated using indicators such as confusion matrix, accuracy, precision, recall, and F1 score. The classification results are fed back to the user or integrated into other systems for further analysis or decision-making.

[0057] In one embodiment, the model can be continuously adjusted and optimized based on the feedback of the classification results to improve the accuracy and efficiency of the classification. In practical applications, the accuracy of the classification results is affected by many factors, including the training data of the model, the model structure, the hyperparameter setting, the data preprocessing and enhancement strategy, etc. Therefore, continuous model evaluation and optimization are the key to improving the quality of the classification results.

[0058] This embodiment provides an audio classification method based on spectrum transformation. The method of this application converts the audio data to be classified into a target spectrum graph through a signal transformation algorithm, and converts the timing information of the audio signal into frequency domain information, which not only reduces the storage requirements of the audio data, but also converts the continuous waveform data into image data, which is convenient for processing the audio signal through image processing technology. The target spectrum graph is classified and identified using a pre-trained image classification model, which improves the accuracy and efficiency of audio classification. By converting audio processing into image processing, not only the storage efficiency is improved and the storage space requirement is reduced, but also the efficiency and accuracy of audio classification are improved by leveraging the maturity and high efficiency of image recognition technology.

[0059] Please refer to Figure 2 , Figure 2 A schematic diagram of a flow chart of a second embodiment of an audio classification method based on spectrum transformation provided by the present application.

[0060] In this embodiment, Figure 2 As shown, based on the above Figure 1 In the illustrated embodiment, before S103, the following steps are further included:

[0061] S201, obtaining annotated audio data;

[0062] In one embodiment, before model training, it is necessary to collect and prepare data sets for training and testing the model. These data sets contain a large number of audio samples, each of which has a category label for identifying the type of audio.

[0063] Specifically, if the dataset contains a metadata file, the audio file path and category label can be directly extracted from it. If metadata is not available, it is necessary to scan the audio file directory, extract the category label from the file name or subfolder name, and map it to a numeric class ID.

[0064] S202, performing spectrum conversion on the annotated audio data based on the signal conversion algorithm to obtain annotated spectrum data;

[0065] In one embodiment, the process of converting a continuous audio signal into a discrete digital signal is called sampling. The sampling frequency determines the number of sampling points, which affects the quality and data volume of the audio signal. The storage accuracy of the sampled value, i.e., the sampling depth, is determined. Commonly used ones are 16bit or 24bit. The higher the quantization level, the more delicate the sound restoration.

[0066] In one embodiment, the audio signal is framed, windowed, and then Fourier transformed to obtain a spectrogram showing the process of signal frequency changing over time. Framing and windowing are preprocessing steps that help reduce the boundary effect of the signal and make signal processing more localized.

[0067] In one embodiment, considering the different sensitivities of the human ear to different frequencies, the common frequency is converted into the Mel frequency to simulate the human ear's perception of the frequency, so that the spectrogram is more in line with the characteristics of human hearing and the recognizability of the audio features is improved. By taking the logarithm of the Mel spectrum and performing a discrete cosine transform (DCT), coefficients that can represent the speech features are extracted. These coefficients can capture the key information of the audio signal, and due to the compression characteristics of the DCT, the amount of data can be further reduced while retaining the most important feature information. Using the above-mentioned extracted features, a spectrogram of the audio signal can be generated, that is, annotated spectrum data including annotation information, and the audio signal is converted from the time domain to a two-dimensional representation in the frequency domain.

[0068] S203: Based on the labeled spectrum data, perform model training on the pre-trained model to generate the image classification model.

[0069] In one embodiment, the pre-processed and feature-extracted audio data is input into the model for training. During the training process, the model learns to recognize audio features of different categories.

[0070] Specifically, a pre-trained convolutional neural network (CNN) can be used as a base model and fine-tuned, such as modifying the first layer of the model so that it only receives single-channel input to adapt to the classification task of single-channel data (spectrogram). During the training process, the labeled spectrum data is input into the convolutional neural network, and the output category probability is calculated through forward propagation. The model weights are adjusted through optimization algorithms such as backpropagation and gradient descent to minimize the loss function.

[0071] Furthermore, based on the pre-trained model, the labeled spectral data is classified and predicted, and the category prediction probability is output; based on the labeled audio data, the prediction accuracy of the category prediction probability is determined; when the prediction accuracy is less than a preset accuracy threshold, the pre-trained model is optimized until the prediction accuracy is greater than or equal to the preset accuracy threshold, so as to obtain the image classification model.

[0072] In one embodiment, the pre-trained model is fine-tuned using the annotated audio data. The fine-tuning can be comprehensive fine-tuning or parameter efficient fine-tuning (PEFT), which only adjusts some parameters of the model. The fine-tuned model is trained using the annotated data, and classification prediction is performed to output the predicted probability of each category. The prediction accuracy of the model is determined by comparing the model prediction results with the actual labels.

[0073] In one embodiment, after training is completed, the model is used to classify and identify new audio data. The model will output a prediction score for each category, thereby identifying the classification information of the audio. The performance of the model is evaluated using a validation set or a test set, and indicators such as accuracy are usually used to measure the classification effect of the model.

[0074] In one embodiment, the prediction accuracy of the model is calculated by comparing the prediction results of the model with the real labels of the annotated audio data. If the prediction accuracy is lower than a preset threshold, the model needs to be iteratively optimized. The pre-trained model is iteratively optimized through the model optimization strategy until the accuracy requirement is met, that is, the training and evaluation process is repeated until the prediction accuracy of the model is greater than or equal to the preset accuracy threshold.

[0075] Generally, optimization strategies include: adjusting hyperparameters, regularization techniques, learning rate adjustment, data enhancement, etc.

[0076] Among them, hyperparameters such as learning rate and batch size can be adjusted by finding the best combination through grid search, random search or Bayesian optimization. Regularization technology refers to the application of L1, L2 regularization or Dropout techniques to reduce overfitting and improve the generalization ability of the model. Learning rate adjustment can use adaptive learning rate methods such as Adam, or set learning rate decay strategies to accelerate model convergence. Data enhancement refers to improving the generalization ability of the model by increasing the diversity of training samples.

[0077] In one embodiment, based on the labeling information of the labeled audio data, a first category distribution of the labeled audio data is determined; based on the category prediction probability, a second category distribution predicted by the pre-trained model for the labeled audio data is determined; and the first category distribution and the second category distribution are compared to calculate the prediction accuracy of the pre-trained model.

[0078] Specifically, the number or proportion of samples of each category in the annotated audio data is counted. This can be done by counting and normalizing the annotation information. For example, if there are 100 samples in the dataset, of which there are 40 in category A, 30 in category B, and 30 in category C, the first category distribution can be expressed as: P(A) = 0.4, P(B) = 0.3, P(C) = 0.3.

[0079] Specifically, the pre-trained model is used to predict the labeled audio data, and the category prediction probability of each sample is recorded. For each sample, the category with the highest probability is selected as the predicted category. The predicted categories of all samples are counted to obtain the predicted category distribution. This is also done by counting and normalizing. For each sample, if the predicted category is consistent with the actual category, the sample is considered to be correctly classified. The prediction accuracy is the ratio of the number of correctly classified samples to the total number of samples.

[0080] For example, for each sample i, if the predicted category is The true category is y i , then check Perform the above check on all samples and count the number of samples that meet the conditions, that is, the number of correctly predicted samples. Divide the number of correctly predicted samples by the total number of samples to get the accuracy.

[0081] In one embodiment, in addition to calculating the accuracy, the first category distribution and the second category distribution can also be compared to see if there is any deviation in the performance of the model in each category. This can be done by calculating indicators such as precision, recall, and F1 score in each category, so as to more comprehensively evaluate the model performance.

[0082] This embodiment can quantitatively evaluate the performance of the pre-trained model on a specific audio classification task by calculating the prediction accuracy, and provide a basis for the selection and optimization of the model. The calculation results of the prediction accuracy can help identify the categories in which the model performs poorly, so as to optimize the model in a targeted manner, such as by adjusting the model structure, adding data enhancement, or adjusting hyperparameters. If it is found that the accuracy is inconsistent in different categories, methods such as resampling technology or cost-sensitive learning can be used to deal with the problem of class imbalance. By comparing the actual distribution and the predicted distribution, the generalization ability of the model can be verified to ensure the effectiveness of the model in practical applications.

[0083] See also Figure 3 , Figure 3 It is a structural schematic diagram of a first embodiment of an audio classification device based on spectrum transformation provided by the present application. The audio classification device based on spectrum transformation is used to execute the aforementioned audio classification method based on spectrum transformation.

[0084] like Figure 3 As shown, the audio classification device 300 based on spectrum transformation includes: a data acquisition module 301, a spectrum conversion module 302 and a model prediction module 303.

[0085] The data acquisition module 301 is used to acquire audio data to be classified;

[0086] The spectrum conversion module 302 is used to perform signal conversion on the audio data to be classified based on a signal conversion algorithm to generate a target spectrum graph;

[0087] The model prediction module 303 is used to classify and identify the target spectrogram based on a pre-trained image classification model, and output an audio classification result of the target spectrogram.

[0088] In one embodiment, the audio classification device 300 based on spectrum transformation further includes:

[0089] The data conversion unit is used to convert the audio data to be classified into a continuous digital sequence based on the sampling rate of the audio data to be classified, so as to generate the target spectrogram according to the digitally represented audio data to be classified.

[0090] In one embodiment, the spectrum conversion module 302 includes:

[0091] A frequency domain information obtaining unit, configured to perform frequency domain conversion on the audio data to be classified based on the signal conversion algorithm to obtain frequency information;

[0092] The spectrum graph generating unit is used to perform spectrum transformation on the frequency information based on a spectrum transformation algorithm to generate the target spectrum graph.

[0093] In one embodiment, the audio classification device 300 based on spectrum transformation further includes a model training module, including:

[0094] The annotated data acquisition submodule is used to acquire annotated audio data;

[0095] A submodule for obtaining annotated spectrum data, for performing spectrum conversion on the annotated audio data based on the signal conversion algorithm to obtain annotated spectrum data;

[0096] The model training submodule is used to perform model training on the pre-trained model based on the labeled spectrum data to generate the image classification model.

[0097] In one embodiment, the model training submodule includes:

[0098] A category prediction unit, used for category prediction, classifying and predicting the labeled spectrum data based on the pre-trained model, and outputting a category prediction probability;

[0099] A prediction accuracy determination unit, used to determine the prediction accuracy of the category prediction probability based on the labeled audio data;

[0100] The model optimization unit is used to optimize the pre-trained model when the prediction accuracy is less than a preset accuracy threshold, until the prediction accuracy is greater than or equal to the preset accuracy threshold, so as to obtain the image classification model.

[0101] In one embodiment, the prediction accuracy determination unit includes:

[0102] A first category distribution determining subunit, configured to determine a first category distribution of the annotated audio data based on the annotated information of the annotated audio data;

[0103] A second category distribution determination subunit, configured to determine a second category distribution predicted by the pre-trained model for the labeled audio data based on the category prediction probability;

[0104] The prediction accuracy determination subunit is used to compare the first category distribution and the second category distribution and calculate the prediction accuracy of the pre-trained model.

[0105] In one embodiment, the audio classification device 300 based on spectrum transformation further includes:

[0106] A data preprocessing unit is used to perform data processing on the target spectrum graph based on an image preprocessing algorithm so that the target spectrum graph meets the input parameter requirements of the image classification model.

[0107] It should be noted that technicians in the relevant field can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described device and each module can refer to the corresponding process in the aforementioned audio classification method embodiment based on spectrum transformation, and will not be repeated here.

[0108] The apparatus provided in the above embodiment may be implemented in the form of a computer program. The computer program may be Figure 4 Runs on the computer device shown.

[0109] See also Figure 4 , Figure 4 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device may be a server.

[0110] See also Figure 4 The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0111] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any audio classification method based on spectrum transformation.

[0112] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0113] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any audio classification method based on spectrum transformation.

[0114] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 4The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0115] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0116] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:

[0117] Obtain audio data to be classified;

[0118] Based on a signal conversion algorithm, the audio data to be classified is converted into a signal to generate a target spectrum diagram;

[0119] Based on the pre-trained image classification model, the target spectrogram is classified and identified, and the audio classification result of the target spectrogram is output.

[0120] In one embodiment, before implementing the signal conversion algorithm to perform signal conversion on the audio data to be classified to generate a target spectrogram, the processor is further configured to implement:

[0121] Based on the sampling rate of the audio data to be classified, the audio data to be classified is converted into a continuous digital sequence, so as to generate the target spectrogram according to the digitally represented audio data to be classified.

[0122] In one embodiment, when the processor implements the signal conversion algorithm based on the signal conversion algorithm to perform signal conversion on the audio data to be classified and generate a target spectrogram, it is used to implement:

[0123] Based on the signal conversion algorithm, the audio data to be classified is converted into a frequency domain to obtain frequency information;

[0124] Based on a spectrum transformation algorithm, the frequency information is subjected to spectrum transformation to generate the target spectrum graph.

[0125] In one embodiment, before implementing the pre-trained image classification model to classify and identify the target spectrogram and output the audio classification result of the target spectrogram, the processor is further used to implement:

[0126] Get the annotated audio data;

[0127] Based on the signal conversion algorithm, the labeled audio data is subjected to spectrum conversion to obtain labeled spectrum data;

[0128] Based on the labeled spectrum data, the pre-trained model is trained to generate the image classification model.

[0129] In one embodiment, when the processor performs model training on the pre-trained model based on the labeled spectrum data to generate the image classification model, it is used to implement:

[0130] Based on the pre-trained model, classify and predict the labeled spectrum data, and output the category prediction probability;

[0131] Determining the prediction accuracy of the category prediction probability based on the labeled audio data;

[0132] When the prediction accuracy is less than a preset accuracy threshold, the pre-trained model is optimized until the prediction accuracy is greater than or equal to the preset accuracy threshold, thereby obtaining the image classification model.

[0133] In one embodiment, when the processor implements the step of determining the prediction accuracy of the category prediction probability based on the labeled audio data, the processor is configured to implement:

[0134] Determining a first category distribution of the annotated audio data based on the annotated information of the annotated audio data;

[0135] Based on the category prediction probability, determining a second category distribution predicted by the pre-trained model for the labeled audio data;

[0136] Compare the first category distribution with the second category distribution, and calculate the prediction accuracy of the pre-trained model.

[0137] In one embodiment, before implementing the pre-trained image classification model to classify and identify the target spectrogram and output the audio classification result of the target spectrogram, the processor is further used to implement:

[0138] Based on an image preprocessing algorithm, data processing is performed on the target spectrum graph so that the target spectrum graph meets the input parameter requirements of the image classification model.

[0139] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium stores a computer program. The computer program includes program instructions. The processor executes the program instructions to implement any audio classification method based on spectrum transformation provided in the embodiments of the present application.

[0140] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.

[0141] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. An audio classification method based on spectrum transformation, characterized in that: The method comprises: Obtain audio data to be classified; Based on a signal conversion algorithm, the audio data to be classified is converted into a signal to generate a target spectrum diagram; Based on the pre-trained image classification model, the target spectrogram is classified and identified, and the audio classification result of the target spectrogram is output.

2. The audio classification method based on spectrum transformation according to claim 1, characterized in that: Before performing signal conversion on the audio data to be classified based on the signal conversion algorithm to generate the target spectrum graph, the method further includes: Based on the sampling rate of the audio data to be classified, the audio data to be classified is converted into a continuous digital sequence, so as to generate the target spectrogram according to the digitally represented audio data to be classified.

3. The audio classification method based on spectrum transformation according to claim 1, characterized in that: The method of performing signal conversion on the audio data to be classified based on a signal conversion algorithm to generate a target spectrum diagram includes: Based on the signal conversion algorithm, the audio data to be classified is converted into a frequency domain to obtain frequency information; Based on a spectrum transformation algorithm, the frequency information is subjected to spectrum transformation to generate the target spectrum graph.

4. The audio classification method based on spectrum transformation according to claim 1, characterized in that: Before classifying and identifying the target spectrogram based on the pre-trained image classification model and outputting the audio classification result of the target spectrogram, the method further includes: Get the annotated audio data; Based on the signal conversion algorithm, the labeled audio data is subjected to spectrum conversion to obtain labeled spectrum data; Based on the labeled spectrum data, the pre-trained model is trained to generate the image classification model.

5. The audio classification method based on spectrum transformation according to claim 4 is characterized in that: The step of training the pre-trained model based on the labeled spectrum data to generate the image classification model includes: Based on the pre-trained model, classify and predict the labeled spectrum data, and output the category prediction probability; Determining the prediction accuracy of the category prediction probability based on the labeled audio data; When the prediction accuracy is less than a preset accuracy threshold, the pre-trained model is optimized until the prediction accuracy is greater than or equal to the preset accuracy threshold, thereby obtaining the image classification model.

6. The audio classification method based on spectrum transformation according to claim 5 is characterized in that: The step of determining the prediction accuracy of the category prediction probability based on the labeled audio data includes: Determining a first category distribution of the annotated audio data based on the annotated information of the annotated audio data; Based on the category prediction probability, determining a second category distribution predicted by the pre-trained model for the labeled audio data; Compare the first category distribution with the second category distribution, and calculate the prediction accuracy of the pre-trained model.

7. The audio classification method based on spectrum transformation according to claim 1, characterized in that: Before classifying and identifying the target spectrogram based on the pre-trained image classification model and outputting the audio classification result of the target spectrogram, the method further includes: Based on an image preprocessing algorithm, data processing is performed on the target spectrum graph so that the target spectrum graph meets the input parameter requirements of the image classification model.

8. An audio classification device based on spectrum transformation, characterized in that: The audio classification device based on spectrum conversion comprises: A data acquisition module, used to acquire audio data to be classified; A spectrum conversion module, used for performing signal conversion on the audio data to be classified based on a signal conversion algorithm to generate a target spectrum graph; The model prediction module is used to classify and identify the target spectrogram based on a pre-trained image classification model, and output the audio classification result of the target spectrogram.

9. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the audio classification method based on spectrum transformation as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the audio classification method based on spectrum transformation as claimed in any one of claims 1 to 7 are implemented.

Images