Audio classification method and apparatus, electronic device, and storage medium

Abstract

The embodiment of the application provides a kind of audio classification method and device, electronic equipment and storage medium, belong to artificial intelligence technical field.The method comprises: obtaining sample video data;Sample video data is extracted, and sample image data, sample text data, sample audio data are obtained;Through the initial matching model matched in advance, sample audio data, sample text data are semantically matched, and first matching data are obtained;Through the initial matching model matched in advance, sample audio data, sample image data are semantically matched, and second matching data are obtained;According to first matching data, second matching data, the initial matching model is adjusted, and target matching model is obtained;According to target matching model and the initial classification model matched in advance, target classification model is constructed;Through target classification model, target audio data is classified.The embodiment of the application can improve the accuracy of audio data classification.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an audio classification method and apparatus, electronic device and storage medium. Background Technology

[0002] In related technologies, pre-trained machine learning models are used to classify audio data. However, these models are trained based on preset sample audio data and preset sample categories. Therefore, when the target category of the audio data to be identified differs from the sample category, the data to be identified and the target category need to be used as new training samples, and the model needs to be retrained based on these new samples to ensure the accuracy of the model's audio data classification. Therefore, how to improve the accuracy of audio data classification without retraining the model has become a pressing technical problem. Summary of the Invention

[0003] The main objective of this application is to provide an audio classification method, apparatus, electronic device, and storage medium, which aims to improve the accuracy of audio data classification.

[0004] To achieve the above objectives, a first aspect of this application proposes an audio classification method, the method comprising:

[0005] Acquire sample video data;

[0006] Feature extraction is performed on the sample video data to obtain sample image data, sample text data, and sample audio data;

[0007] The sample audio data and the sample text data are semantically matched using a preset initial matching model to obtain first matching data; wherein, the first matching data is used to characterize the matching situation between the sample audio data and the sample text data;

[0008] The initial matching model is used to perform semantic matching on the sample audio data and the sample image data to obtain second matching data; wherein, the second matching data is used to characterize the matching situation between the sample audio data and the sample image data;

[0009] The parameters of the initial matching model are adjusted based on the first matching data and the second matching data to obtain the target matching model;

[0010] Construct a target classification model based on the target matching model and the preset initial classification model;

[0011] The target audio data is classified using the target classification model.

[0012] In some embodiments, the sample text data includes first text data that matches the sample audio data, and also includes second text data that does not match the sample audio data;

[0013] The step of semantically matching the sample audio data and the sample text data using a preset initial matching model to obtain first matching data includes:

[0014] The sample audio data is encoded using the initial matching model to obtain a sample audio vector;

[0015] The first text data is encoded using the initial matching model to obtain a first text vector.

[0016] The second text data is encoded using the initial matching model to obtain the second text vector;

[0017] The first matching data is obtained by performing semantic matching based on the sample audio vector, the first text vector, and the second text vector.

[0018] In some embodiments, the step of performing semantic matching based on the sample audio vector, the first text vector, and the second text vector to obtain the first matching data includes:

[0019] Semantic similarity is calculated based on the first text vector and the second text vector to obtain text similarity data;

[0020] Semantic matching is performed based on the sample audio vector and the first text vector to obtain the first sub-matching data;

[0021] Semantic matching is performed based on the sample audio vector and the second text vector to obtain the second sub-matching data;

[0022] The first matching data is obtained by performing a contrast loss calculation based on the text similarity data, the first sub-matching data, and the second sub-matching data.

[0023] In some embodiments, the initial matching model includes an audio encoder, which includes convolutional layers and fully connected layers;

[0024] The step of encoding the sample audio data using the initial matching model to obtain a sample audio vector includes:

[0025] The sample audio features are obtained by extracting features from the sample audio data through the convolutional layer.

[0026] The fully connected layer is used to transform the dimensions of the sample audio features to obtain the sample audio vector.

[0027] In some embodiments, the sample image data includes first image data that matches the sample audio data, and also includes second image data that does not match the sample audio data;

[0028] Semantic matching is performed on the sample audio data and the sample image data using the initial matching model to obtain second matching data, including:

[0029] The sample audio data is encoded using the initial matching model to obtain a sample audio vector;

[0030] The first image data is encoded using the initial matching model to obtain the first image vector;

[0031] The second image data is encoded using the initial matching model to obtain the second image vector;

[0032] The second matching data is obtained by performing semantic matching based on the sample audio vector, the first image vector, and the second image vector.

[0033] In some embodiments, the step of performing semantic matching based on the sample audio vector, the first image vector, and the second image vector to obtain the second matching data includes:

[0034] Semantic similarity is calculated based on the first image vector and the second image vector to obtain image similarity data;

[0035] Semantic matching is performed based on the sample audio vector and the first image vector to obtain the third sub-matching data;

[0036] Semantic matching is performed based on the sample audio vector and the second image vector to obtain the fourth sub-matching data;

[0037] The second matching data is obtained by performing a contrast loss calculation based on the image similarity data, the third sub-matching data, and the fourth sub-matching data.

[0038] In some embodiments, the step of extracting features from the sample video data to obtain sample image data, sample text data, and sample audio data includes:

[0039] Image frames are extracted from the sample video data to obtain a sample image set;

[0040] One image is randomly extracted from the sample image set to obtain the sample image data;

[0041] Subtitles are extracted from the sample video data to obtain the sample text data;

[0042] Audio is extracted from the sample video data to obtain the sample audio data.

[0043] To achieve the above objectives, a second aspect of this application provides an audio classification apparatus, the apparatus comprising:

[0044] The sample data acquisition module is used to acquire sample video data;

[0045] The feature extraction module is used to extract features from the sample video data to obtain sample image data, sample text data, and sample audio data.

[0046] The first matching module is used to perform semantic matching on the sample audio data and the sample text data using a preset initial matching model to obtain first matching data; wherein, the first matching data is used to characterize the matching situation between the sample audio data and the sample text data;

[0047] The second matching module is used to perform semantic matching on the sample audio data and the sample image data using the initial matching model to obtain second matching data; wherein, the second matching data is used to characterize the matching situation between the sample audio data and the sample image data;

[0048] The parameter adjustment module is used to adjust the parameters of the initial matching model according to the first matching data and the second matching data to obtain the target matching model.

[0049] The model building module is used to build a target classification model based on the target matching model and the preset initial classification model;

[0050] The category recognition module is used to classify the target audio data using the target classification model.

[0051] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0052] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0053] The audio classification method, apparatus, electronic device, and storage medium proposed in this application perform semantic matching on sample audio data and sample text data using an initial matching model to obtain first matching data. Then, based on the initial matching model, semantic matching is performed on sample audio data and sample image data to obtain second matching data. Therefore, when the parameters of the initial matching model are adjusted based on the first and second matching data, a target matching model with data semantic description capabilities can be obtained. A target classification model is constructed using the target matching model and a preset initial classification model, and the target audio data is classified according to this target classification model. Thus, since the target classification model possesses the data semantic description capabilities of the target matching model, it can still accurately describe the semantics of the target audio data even when the category of the target audio data changes. In this case, it is only necessary to adjust the category output by the target classification model to include the possible categories of the target audio, thereby avoiding the situation in related technologies where, when the category of the target audio changes, a new training set needs to be constructed based on the changed category, and the model needs to be retrained based on the constructed training set. Therefore, the embodiments of this application improve the accuracy of the target classification model in classifying audio data without retraining the model. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating an audio classification method according to an embodiment of this application;

[0055] Figure 2 This is another schematic flowchart of the audio classification method according to an embodiment of this application;

[0056] Figure 3 This is another schematic flowchart of the audio classification method according to an embodiment of this application;

[0057] Figure 4 This is another schematic flowchart of the audio classification method according to an embodiment of this application;

[0058] Figure 5 This is another schematic flowchart of the audio classification method according to an embodiment of this application;

[0059] Figure 6 This is another schematic flowchart of the audio classification method according to an embodiment of this application;

[0060] Figure 7 This is another schematic flowchart of the audio classification method according to an embodiment of this application;

[0061] Figure 8 This is a schematic diagram of the structure of the audio classification device provided in the embodiments of this application;

[0062] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0064] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0066] First, let's analyze some of the terms used in this application:

[0067] Artificial intelligence (AI) is a new branch of computer science that studies, develops, and applies theories, methods, technologies, and systems to simulate, extend, and expand human intelligence. It aims to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results.

[0068] Contrastive Language-Image Pre-training (CLIP) is a multimodal model based on contrastive learning. CLIP's training data consists of text-image pairs—an image and its corresponding text description. Through contrastive learning, the CLIP model learns the matching relationships between text-image pairs. The CLIP model comprises two modules: an image encoding module and a text encoding module. The image encoding module extracts image features, using either a Convolutional Neural Network (CNN) or a Vision Transformer. The text encoding module extracts text features, using either a Natural Language Processing (NLP) or Text Transformer. The extracted text and image features are compared and contrasted. For a training set containing N text-image pairs, the CLIP model combines each pair of text and image features to predict N... 2 The similarity of N possible text-image pairs. 2 There are N positive samples in the data, which are the text and image pairs that truly belong to each other, and the remaining N are... 2 -N text-image pairs are considered negative samples. The training objective of the CLIP model is to maximize the similarity of the N positive samples while minimizing N. 2 The similarity of -N negative samples.

[0069] Natural Language Processing (NLP): NLP uses computers to process, understand, and utilize human language (such as Chinese and English). NLP is a branch of artificial intelligence and an interdisciplinary field of computer science and linguistics, often referred to as computational linguistics. NLP includes syntactic analysis, semantic analysis, and discourse understanding. It is commonly used in machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, intent recognition, information extraction and filtering, text classification and clustering, sentiment analysis, and opinion mining. It involves data mining, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research, and linguistic research related to language computation.

[0070] In related technologies, pre-trained machine learning models are used to classify audio data. However, these models are trained based on preset sample audio data and preset sample categories. Therefore, when the target category of the audio data to be identified differs from the sample category, the data to be identified and the target category need to be used as new training samples, and the model needs to be retrained based on these new samples to ensure the accuracy of the model's audio data classification. Therefore, how to improve the accuracy of audio data classification without retraining the model has become a pressing technical problem.

[0071] Based on this, embodiments of this application provide an audio classification method and apparatus, an electronic device and a storage medium, aimed at improving the accuracy of audio data classification.

[0072] The audio classification method, apparatus, electronic device, and storage medium provided in this application are specifically described through the following embodiments. First, the audio classification method in this application is described.

[0073] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0074] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0075] The audio classification method provided in this application relates to the field of artificial intelligence technology. The audio classification method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the audio classification method, but is not limited to the above forms.

[0076] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0077] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, user video data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirects to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments obtained.

[0078] Figure 1 This is an optional flowchart of the audio classification method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S101 to S107.

[0079] Step S101: Obtain sample video data;

[0080] Step S102: Extract features from the sample video data to obtain sample image data, sample text data, and sample audio data;

[0081] Step S103: Perform semantic matching on the sample audio data and sample text data using a preset initial matching model to obtain the first matching data; wherein, the first matching data is used to characterize the matching situation between the sample audio data and the sample text data.

[0082] Step S104: Perform semantic matching on the sample audio data and sample image data using the initial matching model to obtain the second matching data; wherein, the second matching data is used to characterize the matching situation between the sample audio data and the sample image data;

[0083] Step S105: Adjust the parameters of the initial matching model based on the first matching data and the second matching data to obtain the target matching model;

[0084] Step S106: Construct a target classification model based on the target matching model and the preset initial classification model;

[0085] Step S107: Classify the target audio data using a target classification model.

[0086] Steps S101 to S107 of this embodiment involve semantic matching of sample audio data and sample text data using an initial matching model to obtain first matching data. Semantic matching of sample audio data and sample image data is then performed using the initial matching model to obtain second matching data. Therefore, when the parameters of the initial matching model are adjusted based on the first and second matching data, a target matching model with data semantic description capabilities can be obtained. A target classification model is constructed using the target matching model and a preset initial classification model, and the target audio data is classified according to this target classification model. Thus, since the target classification model possesses the data semantic description capabilities of the target matching model, it can still accurately describe the semantics of the target audio data even when the category of the target audio data changes. In this case, it is only necessary to adjust the category output by the target classification model to include possible categories of the target audio, thus avoiding the situation in related technologies where, when the category of the target audio changes, a new training set needs to be constructed based on the changed category, and the model needs to be retrained based on the constructed training set. Therefore, this embodiment improves the accuracy of the target classification model in classifying audio data without requiring retraining of the model.

[0087] In step S101 of some embodiments, a training dataset is obtained through a video media platform, short video application, etc., and this training dataset includes multiple sample video data. Each sample video data includes video data in three modalities: image, audio, and text. Furthermore, to improve the accuracy of the initial matching model, the playable duration of each sample video data should be less than a preset duration, for example, a preset duration of 10 seconds. Therefore, when the playable duration of a certain video data exceeds the preset duration, the video data can be trimmed, that is, one video data can be trimmed into two sample video data, thereby enhancing the training dataset of the initial matching model to a certain extent. It is understood that the format of the sample video data can be any of AVI, WMV, MPEG, QuickTime, RealVideo, Flash, or MPEG-4 formats; this embodiment of the application does not specifically limit this.

[0088] In step S102 of some embodiments, feature extraction is performed on the sample video data to extract three modalities of data, namely, multiple original image data, sample text data, and sample audio data. It is understood that since the playable duration of the sample video data is less than a preset duration, the semantic similarity of the extracted original image data is high. To improve the training efficiency of the initial matching model, the multiple original image data can be filtered, and one of the filtered original image data can be used as the sample image data. It is understood that the sample text data is the set of all subtitles corresponding to the sample video data, and the sample audio data is the set of all audio files corresponding to the sample video data.

[0089] Reference Figure 2 In some embodiments, step S102 includes, but is not limited to, steps S201 to S204.

[0090] Step S201: Extract image frames from the sample video data to obtain a sample image set;

[0091] Step S202: Randomly extract an image from the sample image set to obtain sample image data;

[0092] Step S203: Extract subtitles from the sample video data to obtain sample text data;

[0093] Step S204: Extract audio from the sample video data to obtain sample audio data.

[0094] In some steps, S201, image frames are extracted from the sample video data to obtain a sample image set containing multiple original images. It is understood that Fast Forward MPEG (FFmpeg) or other methods can be used for image frame extraction, and this embodiment does not specifically limit this. Specifically, the original images can be obtained by setting the frame rate or the cropping interval. Setting the frame rate sets the number of frames captured per second; for example, when the frame rate is set to 25, 25 images will be captured from the original video data per second. The cropping interval sets the start time, frame rate, and duration of the cropping; for example, it can be set to start cropping from the 2nd second of the sample video data, performing the cropping operation at a rate of 25 images per second for a total of 5 seconds. It is understood that this embodiment does not specifically limit the frame rate and cropping interval. The original images can be in any format such as JPEG, TIFF, RAW, or GIF, and this embodiment does not specifically limit this.

[0095] In step S202 of some embodiments, multiple original images in the sample image set are randomly extracted to obtain sample image data. It is understood that, in order to ensure the semantic description capability of the target matching model for the image data, corresponding extraction conditions can be set. For example, the extraction conditions can be conditions related to image resolution, conditions related to image pixel depth, etc., and this application embodiment does not specifically limit these conditions.

[0096] In step S203 of some embodiments, text recognition is performed on the sample video data to extract subtitles from the sample video data, obtaining a set of all subtitles, i.e., obtaining the sample text data. It is understood that text recognition can be performed using methods such as Optical Character Recognition (OCR), and the embodiments of this application are not specifically limited thereto.

[0097] In step S204 of some embodiments, audio is extracted from all video segments of the sample video data by means of video editing software in related technologies, changing the file extension of the corresponding video data, etc., to obtain sample audio data.

[0098] In step S103 of some embodiments, an initial matching model based on the CLIP model is pre-set. Unlike the CLIP model, the initial matching model adds modules for learning audio-image pair matching relationships and audio-text pair matching relationships. That is, the initial matching model uses audio data as a connecting medium between image and text data, thereby associating the three modalities of audio, text, and image. Multiple extracted sample audio data and multiple sample text data are used as input data to the initial matching model in batches. Semantic matching is performed on the sample audio data and sample text data according to the initial matching model to obtain multiple first matching data. The first matching data is used to characterize the matching status between a sample audio data and a sample text data. It can be understood that when the sample audio data and sample text data describe the same thing, it indicates that the sample audio data and sample text data match, or have a high degree of matching. For example, the sample audio data describes the roar of a tiger, and the content of the sample text data is "a tiger is roaring." When the sample audio data and sample text data describe different things, it indicates that the sample audio data and sample text data do not match or have a low degree of matching. For example, the sample audio data describes the roar of a tiger, while the sample text data describes "a rabbit making a sound".

[0099] Reference Figure 3 In some embodiments, the sample text data includes first text data that matches the sample audio data, and the sample text data also includes second text data that does not match the sample audio data. Step S103 includes, but is not limited to, steps S301 to S304.

[0100] Step S301: Encode the sample audio data using the initial matching model to obtain the sample audio vector;

[0101] Step S302: Encode the first text data using the initial matching model to obtain the first text vector;

[0102] Step S303: Encode the second text data using the initial matching model to obtain the second text vector;

[0103] Step S304: Perform semantic matching based on the sample audio vector, the first text vector, and the second text vector to obtain the first matching data.

[0104] It should be noted that, taking a batch unit comprising M sample text data, with any one sample audio data as the data to be matched, as an example, the sample text data in a batch unit includes one first text data that matches the sample audio data. That is, the first text data is text data from the same sample video data source as the sample audio data. For example, the sample audio data is audio data extracted from video data A, and the first text data is text data extracted from subtitles of video data A. The sample text data in the same batch unit also includes M-1 second text data that do not match the sample audio data. That is, the second text data is text data from sample video data sourced differently from the sample audio data. For example, the second text data is text data extracted from subtitles of video data B.

[0105] In step S301 of some embodiments, the initial matching model includes an audio encoder, which encodes the sample audio data to obtain the corresponding audio vector. It is understood that the audio encoder may be a feature extraction model such as the VGGish model in the Super-Resolution Test Sequence (Visual Geometry Group, VGG) model; however, this embodiment does not specifically limit its use.

[0106] Reference Figure 4 In some embodiments, the initial matching model includes an audio encoder, which includes convolutional layers and fully connected layers. Step S301 includes, but is not limited to, steps S401 to S402.

[0107] Step S401: Extract features from the sample audio data using a convolutional layer to obtain sample audio features;

[0108] Step S402: Perform dimensional transformation on the sample audio features through a fully connected layer to obtain the sample audio vector.

[0109] In step S401 of some embodiments, the audio encoder includes fifteen convolutional layers, each with a 3*3 kernel size. Feature extraction from low-level to high-level features is performed on the sample audio data through multiple convolutional layers to obtain sample audio features. It is understood that, compared to low-level features, high-level features contain more audio semantic information. It is also understood that the number of convolutional layers and the kernel size of the audio encoder can be adaptively adjusted according to actual needs, and this application embodiment does not specifically limit this.

[0110] In step S402 of some embodiments, the audio encoder further includes three fully connected layers. These fully connected layers perform dimensionality transformation on the extracted sample audio features, that is, they integrate the multiple sample audio features extracted by the convolutional layers to obtain a 128-dimensional audio embedding, i.e., a sample audio vector. It is understood that the dimension of the sample audio vector can be adaptively adjusted according to actual conditions, and this application does not impose specific limitations on it.

[0111] It is understandable that, besides using an audio encoder to encode the sample audio data, the original image encoder in the CLIP model can also be used to encode the sample audio data. This involves converting the sample audio data into a Mel spectrum, and then encoding the Mel spectrum using the image encoder to obtain the sample audio vector. However, since the Mel spectrum is single-channel data, while image data consists of three RGB channels, obtaining the sample audio vector through the Mel spectrum would alter the structure of the trained image encoder. Therefore, this embodiment uses an audio encoder to encode the sample audio data, avoiding modifications to the original image encoder in the CLIP model, thereby improving the training efficiency of the initial matching model.

[0112] In steps S302 to S303 of some embodiments, the initial matching model further includes a text encoder, which encodes the first text data to obtain a first text vector. The text encoder then encodes the second text data to obtain a second text vector. It is understood that the text encoder may be a feature extraction model such as a neural network model based on Long Short-Term Memory (LSTM), and this embodiment of the application does not specifically limit this.

[0113] In step S304 of some embodiments, semantic matching is performed by comparing the sample audio vector with the first text vector and the second text vector to determine the matching status of the sample audio vector and the first text vector, and the difference between the matching status of the sample audio vector and the second text vector, thereby obtaining the first matching data.

[0114] This application embodiment obtains first matching data that characterizes the differences in text semantic matching by comparing sample audio vectors, first text vectors, and second text vectors. This enables the subsequent adjustment of parameters of the initial matching model based on the first matching data to obtain a target matching model that has a more accurate semantic description capability for text data and audio data.

[0115] Reference Figure 5 In some embodiments, step S304 includes, but is not limited to, steps S501 to S504.

[0116] Step S501: Calculate semantic similarity based on the first text vector and the second text vector to obtain text similarity data;

[0117] Step S502: Perform semantic matching based on the sample audio vector and the first text vector to obtain the first sub-matching data;

[0118] Step S503: Perform semantic matching based on the sample audio vector and the second text vector to obtain the second sub-matching data;

[0119] Step S504: Calculate the contrast loss based on the text similarity data, the first sub-matching data, and the second sub-matching data to obtain the first matching data.

[0120] In step S501 of some embodiments, sample audio data in a batch unit may contain sample audio data with similar audio tracks, resulting in semantic similarity of the corresponding sample text data. Therefore, in order to improve the semantic description ability of the subsequent target matching model for similar sample text data, the semantic similarity of the first text vector and the second text vector is calculated using a preset content-aware loss function to obtain text similarity data. Specifically, the text similarity data W1 is calculated according to the following formula (1). i .

[0121]

[0122] Where M represents the number of sample text data in a batch unit; t1 i t1 represents the first text vector; j Let represent the second text vector; k represents the temperature coefficient. The semantic similarity of a first text vector corresponding to a sample audio data point with M-1 second text vectors within the same batch is calculated, resulting in multiple similarity data points. These multiple similarity data points are then averaged to obtain the text similarity data W1 for the M sample text data points within that batch. i It is understood that similarity calculation can use cosine similarity or other similarity calculation methods, and this application does not specifically limit the methods used in the embodiments.

[0123] In step S502 of some embodiments, the sample audio vector is semantically matched with the first text vector to obtain first sub-matching data representing the positive sample matching situation.

[0124] In step S503 of some embodiments, the sample audio vector is semantically matched with the second text vector to obtain second sub-matching data representing the negative sample matching situation.

[0125] In step S504 of some embodiments, the contrast loss is calculated on the sample audio vector, the first sub-matching data, and the second sub-matching data according to the following formula (2) to obtain the first matching data L representing the semantic matching difference between positive and negative samples. a→t .

[0126]

[0127] Among them, Z a,i Represents the sample audio vector. Represents the first text vector. Let represent the second text vector, and τ represent the temperature coefficient. Therefore, This indicates the first sub-match data. This indicates the second sub-match data.

[0128] In this embodiment, the first matching data is obtained by comparing the loss of the first sub-matching data and the second sub-matching data. This enables the target matching model to be more accurate in describing the semantics of text data and audio data when the parameters of the initial matching model are adjusted based on the first matching data.

[0129] In step S104 of some embodiments, the extracted sample audio data and sample image data are used as input data for an initial matching model in batches. Semantic matching is then performed on the sample audio data and sample image data according to the initial matching model to obtain multiple second matching data. The second matching data characterizes the matching status between a sample audio data and a sample image data. It is understood that when the sample audio data and sample image data describe the same thing, it indicates that the sample audio data and sample image data match, or have a high degree of matching; for example, the sample audio data describes the roar of a tiger, and the sample image data describes a tiger. When the sample audio data and sample image data describe different things, it indicates that the sample audio data and sample image data do not match, or have a low degree of matching; for example, the sample audio data describes the roar of a tiger, and the sample image data describes a rabbit.

[0130] Reference Figure 6 In some embodiments, the sample image data includes an image data that matches the sample audio data, and the sample image data also includes a second image data that does not match the sample audio data. Step S104 includes, but is not limited to, steps S601 to S604.

[0131] Step S601: Encode the sample audio data using the initial matching model to obtain the sample audio vector;

[0132] Step S602: Encode the first image data using the initial matching model to obtain the first image vector;

[0133] Step S603: Encode the second image data using the initial matching model to obtain the second image vector;

[0134] Step S604: Perform semantic matching based on the sample audio vector, the first image vector, and the second image vector to obtain the second matching data.

[0135] It should be noted that, taking a batch unit comprising M sample image data and any sample audio data as the baseline data as an example, the sample image data in a batch unit includes one first image data that matches the sample audio data. That is, the first image data is an image data sourced from the same sample video data as the sample audio data. For example, the sample audio data is the audio data obtained by extracting audio from video data A, and the first image data is any image data obtained by extracting image frames from video data A. The sample image data in the same batch unit also includes M-1 second image data that do not match the sample audio data. That is, the second image data is image data from sample video data sourced from a different source than the sample audio data. For example, the second image data is any image data obtained by extracting image frames from video data C.

[0136] In step S601 of some embodiments, the initial matching model includes an audio encoder, which encodes the sample audio data to obtain the corresponding audio vector. It is understood that the audio encoder may be a feature extraction model such as the VGGish model, and this embodiment of the application does not specifically limit its use.

[0137] In steps S602 to S603 of some embodiments, the initial matching model further includes an image encoder, which encodes the first image data to obtain a first image vector. The image encoder then encodes the second image data to obtain a second image vector. It is understood that the image encoder may employ feature extraction models such as Histogram of Oriented Gradients (HOG) or Local Binary Pattern (LBP), and this embodiment does not specifically limit its application to these models.

[0138] In step S604 of some embodiments, semantic matching is performed by comparing the sample audio vector with the first image vector and the second image vector to determine the matching status of the sample audio vector and the first image vector, and the difference between the matching status of the sample audio vector and the second image vector, to obtain second matching data.

[0139] This application embodiment obtains second matching data that characterizes the semantic matching differences by comparing sample audio vectors, first image vectors, and second image vectors. This enables the subsequent parameter adjustment of the initial matching model based on the second matching data to obtain a target matching model with more accurate semantic description capabilities for image data and audio data.

[0140] Reference Figure 7 In some embodiments, step S604 includes, but is not limited to, steps S701 to S704.

[0141] Step S701: Calculate semantic similarity based on the first image vector and the second image vector to obtain image similarity data;

[0142] Step S702: Perform semantic matching based on the sample audio vector and the first image vector to obtain the third sub-matching data;

[0143] Step S703: Perform semantic matching based on the sample audio vector and the second image vector to obtain the fourth sub-matching data;

[0144] Step S704: Calculate the contrast loss based on the image similarity data, the third sub-matching data, and the fourth sub-matching data to obtain the second matching data.

[0145] In step S701 of some embodiments, some image frames in a batch of sample audio data may be similar, resulting in semantic similarity of the corresponding sample image data. Therefore, in order to improve the semantic description ability of the subsequent target matching model for similar sample image data, the semantic similarity of the first image vector and the second image vector is calculated using a preset content-aware loss function to obtain image similarity data. Specifically, the image similarity data W2 is calculated according to the following formula (3). i .

[0146]

[0147] Where M represents the number of sample image data in a batch unit, which means that the amount of sample image data, sample audio data, and sample text data in a batch unit is the same; t2 i t2 represents the first image vector; j Let represent the second image vector; k represents the temperature coefficient. The semantic similarity of a first image vector corresponding to a sample audio data point with M-1 second image vectors within the same batch is calculated, resulting in multiple similarity data points. These multiple similarity data points are then averaged to obtain image similarity data W2 for the M sample image data points within the batch. iIt is understood that similarity calculation can use cosine similarity or other similarity calculation methods, and this application does not specifically limit the methods used in the embodiments.

[0148] In step S702 of some embodiments, the sample audio vector is semantically matched with the first image vector to obtain third sub-matching data that represents the positive sample matching situation.

[0149] In step S703 of some embodiments, the sample audio vector is semantically matched with the second image vector to obtain fourth sub-matching data representing the negative sample matching situation.

[0150] In step S704 of some embodiments, the contrast loss is calculated on the sample audio vector, the third sub-matching data, and the fourth sub-matching data according to the following formula (4) to obtain the second matching data L representing the semantic matching difference between positive and negative samples. a→p .

[0151]

[0152] Among them, Z a,i Represents the sample audio vector. Represents the first image vector. Let represent the second image vector, and τ represent the temperature coefficient. Therefore, This indicates the third sub-match data. This represents the fourth sub-match data.

[0153] In this embodiment, the second matching data is obtained by comparing the loss of the third and fourth sub-matching data. This enables the target matching model to be more accurate in terms of semantic description of image data and speech description of audio data when the parameters of the initial matching model are adjusted based on the second matching data.

[0154] In step S105 of some embodiments, the parameters of the initial matching model are adjusted according to the first matching data and the second matching data to improve the semantic description ability of the initial matching model for image data, audio data, and text data. Even if the initial matching model does not focus on data categories, but learns how to mine the essential semantics of data, the target matching model is obtained.

[0155] In step S106 of some embodiments, a target classification model is constructed based on the trained target matching model and a preset initial classification model. It is understood that in the target classification model, the task of the target matching model is to semantically describe the input data, obtaining output data that accurately describes the essential semantics of the input data. The task of the initial classification model is to take the output data of the target matching model as input data and output the category of the input data. For example, when audio data is used as input data for the target classification model, the target matching model in the target classification model performs data mining on the audio data to obtain audio features that describe the essential semantics of the audio data. The initial classification model then classifies these audio features and outputs the category data corresponding to the audio data. It is understood that this category data is used to describe the distribution of the audio data within preset categories. For example, preset categories may include categories such as rock music, folk music, and electronic music, and may also include categories such as tiger roars, rabbit sounds, and dog barks.

[0156] Understandably, the preset categories are pre-set based on the possible categories of the audio data to be identified. For example, if the audio data to be identified may be the sound of a chick, then the preset categories will be set to include the categories of tiger roars, rabbit sounds, dog barks, and chick sounds.

[0157] Understandably, the preset categories are used to constrain the initial category model. Since the target matching model can accurately describe the semantic information of the input data, when the possible categories of the data to be identified change, it is only necessary to adjust the initial classification model according to the changed preset categories. This avoids the situation in related technologies where it is necessary to set training data corresponding to the changed preset categories and retrain the model based on that training data.

[0158] In step S107 of some embodiments, the target audio to be identified is acquired, and the target audio is used as input data for a target classification model to classify the target audio data and obtain category data. The category data is used to characterize the distribution of the target audio data within a preset category.

[0159] This application embodiment performs semantic matching on sample audio data and sample text data using an initial matching model to obtain first matching data. Then, it performs semantic matching on sample audio data and sample image data using the initial matching model to obtain second matching data. Therefore, when the parameters of the initial matching model are adjusted based on the first and second matching data, a target matching model with data semantic description capabilities can be obtained. A target classification model is constructed using the target matching model and a preset initial classification model, and the target audio data is classified according to this target classification model. Thus, since the target classification model possesses the data semantic description capabilities of the target matching model, it can still accurately describe the semantics of the target audio data even when the category of the target audio data changes. In this case, it is only necessary to adjust the category output by the target classification model to include the possible categories of the target audio, thus avoiding the situation in related technologies where, when the category of the target audio changes, a new training set needs to be constructed based on the changed category, and the model needs to be retrained based on the constructed training set. Therefore, this application embodiment improves the accuracy of the target classification model in classifying audio data without requiring model retraining. By comparing and calculating the loss between the first and second sub-matching data, we obtain the first matching data. This allows for subsequent parameter adjustments to the initial matching model based on the first matching data, resulting in a more accurate target matching model that describes the semantics of text data and the speech of audio data. Similarly, by comparing and calculating the loss between the third and fourth sub-matching data, we obtain the second matching data. This allows for subsequent parameter adjustments to the initial matching model based on the second matching data, resulting in a more accurate target matching model that describes the semantics of image data and the speech of audio data.

[0160] Please see Figure 8 This application also provides an audio classification device that can implement the above-described audio classification method. The device includes:

[0161] The sample data acquisition module 801 is used to acquire sample video data;

[0162] The feature extraction module 802 is used to extract features from the sample video data to obtain sample image data, sample text data, and sample audio data.

[0163] The first matching module 803 is used to perform semantic matching on sample audio data and sample text data using a preset initial matching model to obtain first matching data; wherein, the first matching data is used to characterize the matching situation between sample audio data and sample text data.

[0164] The second matching module 804 is used to perform semantic matching on the sample audio data and sample image data through the initial matching model to obtain the second matching data; wherein, the second matching data is used to characterize the matching situation between the sample audio data and the sample image data;

[0165] The parameter adjustment module 805 is used to adjust the parameters of the initial matching model based on the first matching data and the second matching data to obtain the target matching model.

[0166] Model building module 806 is used to build a target classification model based on the target matching model and the preset initial classification model;

[0167] The category recognition module 807 is used to classify target audio data using a target classification model.

[0168] The specific implementation of this audio classification device is basically the same as the specific implementation of the audio classification method described above, and will not be repeated here.

[0169] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned audio classification method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0170] Please see Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0171] The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0172] The memory 902 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the audio classification method of the embodiments of this application.

[0173] The input / output interface 903 is used to implement information input and output;

[0174] The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0175] Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904);

[0176] The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0177] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described audio classification method.

[0178] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0179] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0180] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0181] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0182] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0183] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0184] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0185] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0186] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0187] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0188] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0189] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. An audio classification method, characterized in that, The method includes: Acquire sample video data; Feature extraction is performed on the sample video data to obtain sample image data, sample text data, and sample audio data; The sample audio data and the sample text data are semantically matched using a preset initial matching model to obtain first matching data; wherein, the first matching data is used to characterize the matching situation between the sample audio data and the sample text data; The initial matching model is used to perform semantic matching on the sample audio data and the sample image data to obtain second matching data; wherein, the second matching data is used to characterize the matching situation between the sample audio data and the sample image data; The parameters of the initial matching model are adjusted based on the first matching data and the second matching data to improve the semantic description capabilities of the initial matching model for image data, audio data, and text data, thereby obtaining the target matching model. A target classification model is constructed based on the target matching model and the preset initial classification model; wherein, in the target classification model, the task of the target matching model is to perform semantic description on the input data and obtain output data describing the essential semantics of the input data; the task of the initial classification model is to take the output data of the target matching model as input data and output the category of the input data; The target audio data is classified by the target classification model to obtain the category data of the target audio data. The category data is used to describe the distribution of the target audio data in the preset categories. The preset categories are used to constrain the initial category model. The sample text data includes first text data that matches the sample audio data, and second text data that does not match the sample audio data; the step of semantically matching the sample audio data and the sample text data using a preset initial matching model to obtain the first matching data includes: The sample audio data is encoded using the initial matching model to obtain a sample audio vector; The first text data is encoded using the initial matching model to obtain a first text vector. The second text data is encoded using the initial matching model to obtain the second text vector; Semantic matching is performed on the sample audio vector, the first text vector, and the second text vector to obtain the first matching data. Specifically, semantic matching is performed on the sample audio vector based on the first text vector and the second text vector to determine the matching status of the sample audio vector and the first text vector, and the difference between the matching status of the sample audio vector and the second text vector is used to obtain the first matching data.

2. The method according to claim 1, characterized in that, The step of performing semantic matching based on the sample audio vector, the first text vector, and the second text vector to obtain the first matching data includes: Semantic similarity is calculated based on the first text vector and the second text vector to obtain text similarity data; Semantic matching is performed based on the sample audio vector and the first text vector to obtain the first sub-matching data; Semantic matching is performed based on the sample audio vector and the second text vector to obtain the second sub-matching data; The first matching data is obtained by performing a contrast loss calculation based on the text similarity data, the first sub-matching data, and the second sub-matching data.

3. The method according to claim 1, characterized in that, The initial matching model includes an audio encoder, which includes convolutional layers and fully connected layers. The step of encoding the sample audio data using the initial matching model to obtain a sample audio vector includes: The sample audio features are obtained by extracting features from the sample audio data through the convolutional layer. The fully connected layer is used to transform the dimensions of the sample audio features to obtain the sample audio vector.

4. The method according to claim 1, characterized in that, The sample image data includes first image data that matches the sample audio data, and also includes second image data that does not match the sample audio data; Semantic matching is performed on the sample audio data and the sample image data using the initial matching model to obtain second matching data, including: The sample audio data is encoded using the initial matching model to obtain a sample audio vector; The first image data is encoded using the initial matching model to obtain the first image vector; The second image data is encoded using the initial matching model to obtain the second image vector; The second matching data is obtained by performing semantic matching based on the sample audio vector, the first image vector, and the second image vector.

5. The method according to claim 4, characterized in that, The step of performing semantic matching based on the sample audio vector, the first image vector, and the second image vector to obtain the second matching data includes: Semantic similarity is calculated based on the first image vector and the second image vector to obtain image similarity data; Semantic matching is performed based on the sample audio vector and the first image vector to obtain the third sub-matching data; Semantic matching is performed based on the sample audio vector and the second image vector to obtain the fourth sub-matching data; The second matching data is obtained by performing a contrast loss calculation based on the image similarity data, the third sub-matching data, and the fourth sub-matching data.

6. The method according to any one of claims 1 to 5, characterized in that, The step of extracting features from the sample video data to obtain sample image data, sample text data, and sample audio data includes: Image frames are extracted from the sample video data to obtain a sample image set; One image is randomly extracted from the sample image set to obtain the sample image data; Subtitles are extracted from the sample video data to obtain the sample text data; Audio is extracted from the sample video data to obtain the sample audio data.

7. An audio classification device, characterized in that, The device includes: The sample data acquisition module is used to acquire sample video data; The feature extraction module is used to extract features from the sample video data to obtain sample image data, sample text data, and sample audio data. The first matching module is used to perform semantic matching on the sample audio data and the sample text data using a preset initial matching model to obtain first matching data; wherein, the first matching data is used to characterize the matching situation between the sample audio data and the sample text data; The second matching module is used to perform semantic matching on the sample audio data and the sample image data using the initial matching model to obtain second matching data; wherein, the second matching data is used to characterize the matching situation between the sample audio data and the sample image data; The parameter adjustment module is used to adjust the parameters of the initial matching model according to the first matching data and the second matching data, so as to improve the semantic description ability of the initial matching model for image data, audio data and text data, and obtain the target matching model. The model building module is used to build a target classification model based on the target matching model and the preset initial classification model. In the target classification model, the task of the target matching model is to perform semantic description on the input data and obtain output data that describes the essential semantics of the input data. The task of the initial classification model is to take the output data of the target matching model as input data and output the category of the input data. The category recognition module is used to classify the target audio data using the target classification model to obtain the category data of the target audio data. The category data is used to describe the distribution of the target audio data in a preset category. The preset category is used to constrain the initial category model. The sample text data includes first text data that matches the sample audio data, and second text data that does not match the sample audio data; the step of semantically matching the sample audio data and the sample text data using a preset initial matching model to obtain the first matching data includes: The sample audio data is encoded using the initial matching model to obtain a sample audio vector; The first text data is encoded using the initial matching model to obtain a first text vector. The second text data is encoded using the initial matching model to obtain the second text vector; Semantic matching is performed on the sample audio vector, the first text vector, and the second text vector to obtain the first matching data. Specifically, semantic matching is performed on the sample audio vector based on the first text vector and the second text vector to determine the matching status of the sample audio vector and the first text vector, and the difference between the matching status of the sample audio vector and the second text vector is used to obtain the first matching data.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the audio classification method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the audio classification method according to any one of claims 1 to 6.

Images