Audio recognition method and device, computer program product and electronic equipment

Abstract

The embodiment of the invention relates to the technical field of computers, in particular to an audio recognition method and device, a computer program product and electronic equipment. The method comprises the following steps: acquiring a to-be-recognized audio, and preprocessing the to-be-recognized audio to obtain a to-be-recognized audio clip corresponding to the to-be-recognized audio; obtaining a first recognition model, and inputting the to-be-recognized audio clip into the first recognition model to obtain a first recognition result; and when the first recognition result is a preset result, obtaining a second recognition model, and inputting the to-be-recognized audio clip into the second recognition model to obtain a second recognition result of the to-be-recognized audio. According to the method, the recognition efficiency of the to-be-recognized audio is improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of computer technology, and more specifically, relate to an audio recognition method and device, a computer program product, and an electronic device. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the disclosure that are recited in the claims, and no description herein is admitted to be prior art by inclusion in this section.

[0003] In the related art, when recognizing AI audio, first, audio sample data sets of different styles are obtained and preprocessed; then, multiple features are extracted from the preprocessed audio sample data sets, and the extracted features are weighted fused to obtain an initial fused feature vector, which is input into the initialized model, and an improved Gaussian radial basis function is used as the activation function. The model training is optimized by the gradient descent method until the global optimal solution is reached, thereby obtaining a trained AI audio recognition model, which is evaluated to obtain an evaluated model; finally, the audio data is input into the evaluated model to obtain the recognition result. Summary of the invention

[0004] The above audio recognition method extracts multiple features, performs weighted fusion on the features, and then sends them to the neural network model for training. Since the extraction of each type of feature requires the use of other models or systems, the calculation complexity is high. In addition, the recognition errors of other models will form error accumulation, affecting the final performance of the system. In addition, the fidelity of audio generated by AI is getting higher and higher, and the use of explicit features of audio to identify audio will gradually become ineffective.

[0005] Therefore, there is a great need for an improved audio recognition method and device, a computer program product and an electronic device to provide a method that can significantly improve the recognition accuracy of the audio to be recognized, so as to solve the problem of low recognition accuracy of the audio to be recognized in the related art.

[0006] In this context, embodiments of the present disclosure are intended to provide an audio recognition method and apparatus, a computer program product, and an electronic device.

[0007] According to one aspect of the present disclosure, there is provided an audio recognition method, comprising:

[0008] Acquire the audio to be recognized, and preprocess the audio to be recognized to obtain an audio segment to be recognized corresponding to the audio to be recognized;

[0009] Acquire a first recognition model, input the audio segment to be recognized into the first recognition model, and obtain a first recognition result;

[0010] When the first recognition result is a preset result, a second recognition model is obtained, and the audio segment to be recognized is input into the second recognition model to obtain a second recognition result of the audio segment to be recognized.

[0011] In an exemplary embodiment of the present disclosure, the preprocessing of the audio to be recognized to obtain an audio segment corresponding to the audio to be recognized includes:

[0012] Convert the audio to be recognized into a target format, and segment the audio to be recognized in the target format to obtain a first audio segment;

[0013] Silence detection is performed on the first audio segment to obtain a silence segment included in the first audio segment, and the silence segment is deleted to obtain the audio segment to be identified.

[0014] In an exemplary embodiment of the present disclosure, before acquiring the first recognition model, the method further includes:

[0015] Acquire a first feature extractor and a first classifier, and construct the first recognition model based on the first feature extractor and the first classifier;

[0016] Acquire sample audio, use the sample audio to train the first recognition model, obtain a first output result of the first recognition model, and determine a first loss function of the first recognition model using the first output result;

[0017] Parameters of the first recognition model are updated based on the first loss function.

[0018] In an exemplary embodiment of the present disclosure, the step of inputting the audio segment to be recognized into the first recognition model to obtain a first recognition result includes:

[0019] Inputting the audio segment to be recognized into the first recognition model to obtain a prediction score corresponding to the audio segment to be recognized;

[0020] Obtaining a first prediction score of the audio to be identified according to the prediction score of the audio segment to be identified, and determining that the audio to be identified is a first type of audio when the first prediction score is greater than a first preset value;

[0021] When the first prediction score is less than a second preset value, determining that the audio to be identified is a second type of audio;

[0022] When the first prediction score is not less than the second preset value and not greater than the first preset value, it is determined that the audio to be identified is the third type of audio.

[0023] In an exemplary embodiment of the present disclosure, when the first recognition result is a preset result, obtaining a second recognition model, inputting the audio segment to be recognized into the second recognition model, and obtaining a second recognition result of the audio segment to be recognized includes:

[0024] When the audio to be recognized is the third type of audio, obtaining the second recognition model;

[0025] The audio segment to be recognized of the audio to be recognized is input into the second recognition model to obtain the second recognition result.

[0026] In an exemplary embodiment of the present disclosure, before acquiring the second recognition model, the method further includes:

[0027] Acquire a second feature extractor and a second classifier, and construct the second recognition model based on the second feature extractor and the second classifier;

[0028] Acquire sample audio, input the sample audio into the second feature extractor to obtain hidden features of the sample audio, input the hidden features into the second classifier to obtain a second output result of the sample audio;

[0029] A second loss function is determined according to the second output result, and parameters of the second classifier are updated based on the second loss function.

[0030] In an exemplary embodiment of the present disclosure, the step of inputting the audio segment to be recognized into the second recognition model to obtain a second recognition result of the audio segment to be recognized includes:

[0031] Inputting the audio segment to be identified into the second feature extractor to obtain the hidden features of the audio segment to be identified;

[0032] Inputting the hidden layer features into the second classifier to obtain a prediction score corresponding to the audio segment to be identified;

[0033] A second prediction score of the audio segment to be recognized is obtained according to the prediction score of the audio segment to be recognized, and the second recognition result is obtained according to the second prediction score.

[0034] In an exemplary embodiment of the present disclosure, obtaining the second recognition result according to the second prediction score includes:

[0035] When the second prediction score is greater than a third preset value, determining that the audio to be identified is a first type of audio;

[0036] When the second prediction score is less than a fourth preset value, determining that the audio to be identified is a second type of audio;

[0037] When the second prediction score is not less than the fourth preset value and not greater than the third preset value, obtaining a third prediction score of the audio to be recognized according to the first prediction score of the audio to be recognized and the second prediction score;

[0038] The type of the audio to be recognized is determined according to the third prediction score of the audio to be recognized.

[0039] In an exemplary embodiment of the present disclosure, determining the type of the audio to be recognized according to the third prediction score of the audio to be recognized includes:

[0040] When the third prediction score is greater than the fifth prediction value, determining that the audio to be identified is the first type of audio;

[0041] When the third prediction score is not greater than the fifth prediction value, it is determined that the audio to be identified is the second type of audio.

[0042] According to one aspect of the present disclosure, there is provided an audio recognition device, comprising:

[0043] An audio processing module, used to obtain the audio to be recognized, pre-process the audio to be recognized, and obtain an audio segment to be recognized corresponding to the audio to be recognized;

[0044] A first recognition module, used to obtain a first recognition model, input the audio segment to be recognized into the first recognition model, and obtain a first recognition result;

[0045] The second recognition module is used to obtain a second recognition model when the first recognition result is a preset result, input the audio segment to be recognized into the second recognition model, and obtain a second recognition result of the audio segment to be recognized.

[0046] According to one aspect of the present disclosure, a computer storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned audio recognition method is implemented.

[0047] According to one aspect of the present disclosure, there is provided an electronic device, including:

[0048] A processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform any of the above-mentioned audio recognition methods by executing the executable instructions.

[0049] According to the audio recognition method of the embodiment of the present disclosure, on the one hand, when the audio to be recognized is obtained, the audio to be recognized is preprocessed to obtain an audio segment to be recognized, and the audio segment to be recognized is input into the first recognition model to obtain a first recognition result. After obtaining the first recognition result, the first recognition result is judged. When the first recognition result is a preset result, the audio to be recognized is input into the second recognition model to obtain a second recognition result. Through the collaborative work of the two models, the difference between AI audio and real audio can be fully captured, and the recognition accuracy of the audio to be recognized is improved. On the other hand, after the audio to be recognized is obtained, the audio to be recognized is preprocessed to generate an audio segment to be recognized of the audio to be recognized, and the audio segment to be recognized is directly input into the model, which reduces the length of data dependence and reduces the complexity of data processing. When making predictions, the first recognition model is first used for recognition, and then the second recognition model is used for verification. Under the premise of ensuring accuracy, the recognition time is shortened. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:

[0051] Figure 1 A flowchart of an audio recognition method according to an embodiment of the present disclosure is schematically shown;

[0052] Figure 2 A flowchart of a method for preprocessing the audio to be recognized to obtain an audio segment corresponding to the audio to be recognized according to an embodiment of the present disclosure is schematically shown;

[0053] Figure 3 A flowchart of an audio recognition method before acquiring a first recognition model according to an embodiment of the present disclosure is schematically shown;

[0054] Figure 4 A flowchart of a method for inputting the audio segment to be recognized into the first recognition model to obtain a first recognition result according to an embodiment of the present disclosure is schematically shown;

[0055] Figure 5 A flowchart of a method for obtaining a second recognition model, inputting the audio segment to be recognized into the second recognition model, and obtaining a second recognition result of the audio segment to be recognized according to an embodiment of the present disclosure is schematically shown;

[0056] Figure 6 A flowchart of an audio recognition method according to an embodiment of the present disclosure is schematically shown;

[0057] Figure 7 A flowchart schematically shows a method for inputting the audio segment to be recognized into the second recognition model to obtain a second recognition result of the audio segment to be recognized according to an embodiment of the present disclosure;

[0058] Figure 8 A flowchart of a method for obtaining the second recognition result according to the second prediction score according to an embodiment of the present disclosure is schematically shown;

[0059] Fig. 9 A flowchart of a method for determining the type of the audio to be recognized according to the third prediction score of the audio to be recognized according to an embodiment of the present disclosure is schematically shown;

[0060] Fig.10 A block diagram of an audio recognition device according to an embodiment of the present disclosure is schematically shown;

[0061] Fig.11 A schematic diagram of a computer storage medium according to an embodiment of the present disclosure is shown;

[0062] Fig.12 A block diagram of an electronic device according to a disclosed embodiment is schematically shown.

[0063] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0064] The principles and spirit of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present disclosure, and are not intended to limit the scope of the present disclosure in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0065] Those skilled in the art will appreciate that the embodiments of the present disclosure may be implemented as a system, device, apparatus, method or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0066] According to an embodiment of the present disclosure, a data communication method, a data communication device, a computer program product, and an electronic device are provided.

[0067] In this article, when it is necessary to understand, the terms involved are explained as follows:

[0068] VAD (Voice Activity Detection): A technology used to detect whether an audio signal contains speech. Common VAD algorithms include: energy VAD, model VAD;

[0069] RawNet: An end-to-end speech recognition model architecture based on raw speech signals;

[0070] Mert: An acoustic music understanding model based on large-scale self-supervised training.

[0071] AASIST: An open source model framework combining time domain and frequency domain graph neural networks for audio anti-spoofing scenarios.

[0072] In this document, any number of elements in the drawings is used for illustration rather than limitation, and any naming is used only for distinction and does not have any limiting meaning.

[0073] The principle and spirit of the present disclosure are explained in detail below with reference to several representative embodiments of the present disclosure. SUMMARY OF THE INVENTION

[0075] The inventors found that with the continuous development of artificial intelligence technology, AI music generation technology has gradually matured. The release of various AI music automatic generation tools has lowered the threshold for music production, resulting in a sharp increase in the number of AI music. There are great legal risks in the dissemination of these works, and song platforms need to detect these works. AI music generation technology mainly uses machine learning algorithms to learn the laws and characteristics of music from a large amount of music data, thereby automatically generating new music works. Therefore, there must be certain differences between automatically generated AI music and human-created music. Finding these differences and detecting them through deep learning models makes automatic recognition of AI music possible.

[0076] Detecting AI-generated music is a new field with the following challenges:

[0077] 1. The types of AI music are diverse, including: AI-generated pure accompaniment, AI-generated pure singing, AI cover with normal accompaniment, etc.

[0078] 2. AI music has diverse styles, and different music genres and languages ​​can be generated;

[0079] 3. AI music generation technologies are diverse, including generation technologies based on end-to-end large models and generation technologies based on templates;

[0080] 4. Only some parts of the song are generated by AI;

[0081] Existing technical solutions mainly extract multiple explicit music features and then input them into deep learning models for prediction. Existing feature extraction methods and classification algorithms may not be able to completely and accurately identify AI-generated music, and the computational efficiency is low when processing large-scale data. Therefore, how to improve the accuracy and efficiency of AI music recognition is an important issue facing current technology.

[0082] In view of the above content, the basic idea of ​​the present disclosure is to obtain the audio to be recognized, process the audio to be recognized into an audio segment to be recognized, input the audio segment to be recognized into a first recognition model to obtain a first recognition result, and when the first recognition result is a preset result, input the audio to be recognized into a second recognition model to obtain a second recognition result of the audio to be recognized.

[0083] After introducing the basic principles of the present disclosure, various non-limiting embodiments of the present disclosure are described in detail below.

[0084] Exemplary Methods

[0085] Combine the following Figure 1 The audio recognition method according to an exemplary embodiment of the present disclosure is described. Figure 1 , the audio recognition method may include the following steps:

[0086] S110, obtaining audio to be recognized, and preprocessing the audio to be recognized to obtain an audio segment to be recognized corresponding to the audio to be recognized;

[0087] S120, obtaining a first recognition model, inputting the audio segment to be recognized into the first recognition model, and obtaining a first recognition result;

[0088] S130: When the first recognition result is a preset result, obtain a second recognition model, input the audio segment to be recognized into the second recognition model, and obtain a second recognition result of the audio segment to be recognized.

[0089] In the audio recognition method of the embodiment of the present disclosure, on the one hand, when the audio to be recognized is obtained, the audio to be recognized is preprocessed to obtain an audio segment to be recognized, and the audio segment to be recognized is input into the first recognition model to obtain a first recognition result. After obtaining the first recognition result, the first recognition result is judged. When the first recognition result is a preset result, the audio to be recognized is input into the second recognition model to obtain a second recognition result. Through the collaborative work of the two models, the difference between AI audio and real audio can be fully captured, and the recognition accuracy of the audio to be recognized is improved. On the other hand, after the audio to be recognized is obtained, the audio to be recognized is preprocessed to generate an audio segment to be recognized of the audio to be recognized, and the audio segment to be recognized is directly input into the model, which reduces the length of data dependence and reduces the complexity of data processing. When making predictions, the first recognition model is first used for recognition, and then the second recognition model is used for verification. Under the premise of ensuring accuracy, the recognition time is shortened.

[0090] The audio recognition method is further explained and illustrated below.

[0091] In step S110, the audio to be recognized is obtained, and the audio to be recognized is preprocessed to obtain an audio segment to be recognized corresponding to the audio to be recognized.

[0092] The audio to be identified may be music audio, which may be generated based on AI or may be normal music audio, and the audio to be identified is not specifically limited in this example embodiment. The preprocessing of the audio to be identified may be converting the audio to be identified into a target format, segmenting the audio to be identified converted into the target format, and obtaining an audio segment to be identified, wherein the length of the audio segment to be identified may be 5 seconds or 6 seconds, and this is not specifically limited in this example embodiment.

[0093] In this example embodiment, reference Figure 2 As shown, the preprocessing of the audio to be recognized to obtain an audio segment corresponding to the audio to be recognized includes:

[0094] Step S210: converting the audio to be recognized into a target format, and segmenting the audio to be recognized in the target format to obtain a first audio segment;

[0095] Step S220: Perform silence detection on the first audio segment to obtain a silence segment included in the first audio segment, and delete the silence segment to obtain the audio segment to be identified.

[0096] In the following, step S210 and step S220 will be further explained and illustrated. Specifically, first, the audio to be identified is converted into a target format, wherein the target format can be a 16k16bit single-channel wav format, and the target format is not specifically limited in this example embodiment. After the audio to be identified is converted into the target format, a fixed segment window can be used to act on the audio to be identified to obtain a first audio segment of the audio to be detected. After obtaining the first audio segment, silence detection can be performed on the first audio segment, wherein the energy VAD algorithm can be used for detection, and the detection method is not specifically limited in this example embodiment. When the first audio segment is detected to be pure silence, the first audio segment is deleted to obtain the audio segment to be identified of the audio to be identified. It should be supplemented that when the audio to be identified is segmented, when the length of the last segment of the audio to be identified is less than the preset length, the last audio segment is deleted.

[0097] In step S120, a first recognition model is obtained, and the audio segment to be recognized is input into the first recognition model to obtain a first recognition result.

[0098] The first recognition model includes a first feature extractor and a first classifier. The features extracted by the first feature extractor may be traditional spectrum features, including Mel spectrum, Mel frequency cepstral coefficients, linear prediction cepstral coefficients, etc., which are not specifically limited in this example embodiment. Figure 3 As shown, before obtaining the first recognition model, the method further includes:

[0099] Step S310. Obtain a first feature extractor and a first classifier, and construct the first recognition model based on the first feature extractor and the first classifier;

[0100] Step S320. Obtain sample audio, use the sample audio to train the first recognition model, obtain a first output result of the first recognition model, and use the first output result to determine a first loss function of the first recognition model;

[0101] Step S330. Update parameters of the first recognition model based on the first loss function.

[0102] Below, step S310-step S330 will be further explained and illustrated. Specifically, a first feature extractor and a first classifier are obtained, wherein the first feature extractor can be a neural network feature extractor, specifically RawNet, and the first classifier can be a neural network classifier, specifically Resnet, Gru can be selected as the classifier, and the AASIST graph neural network classifier can also be selected. In this example embodiment, the first classifier and the first feature extractor are not specifically limited. After obtaining the first feature extractor and the second classifier, a first recognition model is formed according to the first feature extractor and the first classifier. Obtain sample audio, the number of which is at least 10,000, and the sample audio has a label indicating whether the audio is generated by AI, wherein the sample audio includes multiple audios generated by AI, and the generators of the AI ​​audio are different. After obtaining the sample audio, convert the sample audio into a 16k16bit single-channel wav format, and divide the sample audio into audio segments of fixed length, input the audio segments into a first recognition model, train the first recognition model, obtain a first output result of the first recognition model, and determine a first loss function using the first output result, wherein the first loss function can be a binary focal loss function, use a gradient descent algorithm to return the loss, update the parameters of the first recognition model, and stop training when the loss function does not decrease for multiple consecutive rounds.

[0103] After obtaining the first recognition model, the audio segment to be recognized can be input into the first recognition model. Figure 4 As shown, the step of inputting the audio segment to be recognized into the first recognition model to obtain a first recognition result includes:

[0104] Step S410: input the audio segment to be recognized into the first recognition model to obtain a prediction score corresponding to the audio segment to be recognized;

[0105] Step S420: Obtain a first prediction score of the audio to be identified according to the prediction score of the audio segment to be identified, and when the first prediction score is greater than a first preset value, determine that the audio to be identified is a first type of audio;

[0106] Step S430: when the first prediction score is less than a second preset value, determining that the audio to be identified is a second type of audio;

[0107] Step S440: When the first prediction score is not less than the second preset value and not greater than the first preset value, determine that the audio to be identified is a third type of audio.

[0108] Next, steps S410 - S440 will be further explained and described. Specifically, the segment to be recognized of the audio to be recognized is input into the first recognition model to obtain a predicted score corresponding to the segment to be recognized. This predicted score is the probability that the segment to be recognized of the audio to be recognized is an AI audio, and it is a value within the range of 0 - 1. After obtaining the predicted score of the segment to be recognized of the audio to be recognized, the first predicted score of the segment to be recognized of the audio to be recognized can be obtained based on the predicted score. Among them, the predicted score of the segment to be recognized of the audio to be recognized can be averaged, the maximum value can be taken, the moving average maximum value can be taken, etc. to obtain the first predicted score. It is also possible to sort the predicted scores of the segment to be recognized of the audio to be recognized from largest to smallest, eliminate 10% of the maximum values and 10% of the minimum values, and average the remaining predicted scores to obtain the first predicted score. In this exemplary embodiment, the calculation method of the first predicted score is not specifically limited. After obtaining the first predicted score, when the first predicted score is greater than the first preset value, it is determined that the audio to be recognized is an AI audio. When the first predicted score is less than the second preset value, it is determined that the audio to be recognized is a second type of audio, and this second type of audio can be a non - AI audio. When the first predicted score is not less than the second preset value and not greater than the first preset value, the audio to be recognized is an audio of uncertain type. Wherein the first preset value is a1, the second preset value is b1, b1 < a1, the value of a1 can be 0.6, and the value of b1 can be 0.4. In this exemplary embodiment, the values of a1 and b1 are not specifically limited.

[0109] In step S130, when the first recognition result is a preset result, a second recognition model is obtained, and the segment to be recognized of the audio to be recognized is input into the second recognition model to obtain the second recognition result of the audio to be recognized.

[0110] Among them, the preset result is that the recognition result of the audio to be recognized is a third type of audio. Refer to Figure 5 As shown, when the first recognition result is a preset result, obtaining a second recognition model and inputting the segment to be recognized of the audio to be recognized into the second recognition model to obtain the second recognition result of the audio to be recognized includes:

[0111] Step S510. When the audio to be recognized is the third type of audio, obtain the second recognition model;

[0112] Step S520. Input the segment to be recognized of the audio to be recognized into the second recognition model to obtain the second recognition result.

[0113] In the following, step S510 and step S520 are further explained and illustrated. Specifically, after obtaining a first recognition result of the audio to be recognized through the first recognition model, when the audio to be recognized is the third type of audio, a second recognition model is obtained, and the audio to be recognized is input into the second recognition model to obtain a second recognition result.

[0114] Further, before obtaining the second recognition model, refer to Figure 6 As shown, the identification method also includes:

[0115] Step S610. Obtain a second feature extractor and a second classifier, and construct the second recognition model based on the second feature extractor and the second classifier;

[0116] Step S620. Obtain sample audio, input the sample audio to the second feature extractor, obtain hidden features of the sample audio, input the hidden features to the second classifier, and obtain a second output result of the sample audio;

[0117] Step S630. Determine a second loss function according to the second output result, and update parameters of the second classifier based on the second loss function.

[0118] Below, step S610-step S630 will be further explained and illustrated. Specifically, a second feature extractor and a second classifier are obtained, and a second recognition model is obtained based on the second feature extractor and the second classifier. Among them, the second feature extractor can be a pre-trained model for music understanding, specifically, it can be Mert or MusicLM, etc., and the second feature extractor is not specifically limited in this example embodiment. The role of the second classifier is to fuse the features of each layer extracted by the second feature extractor. The second classifier can include an attention layer, a pooling layer, and a mapping layer. The attention layer learns the weight coefficients of each layer of the feature, and the sum of the weights of each layer is 1; the pooling layer weights and merges the features of each layer, and uses a 3*3 window for maximum value pooling to reduce the feature dimension; the mapping layer expands the pooling layer features, and the fully connected layer maps the features to 1 dimension and outputs them. After obtaining the second recognition model, obtain sample audio, preprocess the sample audio to obtain an audio segment to be recognized, input the audio segment to be recognized into the second feature extractor of the second recognition model, obtain the hidden layer features of the sample audio segment, input the hidden layer features into the second classifier, obtain the second output result of the sample audio, after obtaining the second output result, determine a second loss function according to the second output result, the second loss function can be a binary focal loss function, and update the parameters of the second classifier based on the second loss function.

[0119] In the present disclosure, an end-to-end deep learning model and a pre-trained music understanding model are adopted. Both models are data-driven optimizations. As long as the training data is increased, the model effect can be quickly iterated and updated. In addition, the first recognition model and the second recognition model can be applicable to audios of various types and styles based on large-scale data training, thereby improving the applicability of audio recognition.

[0120] In this example embodiment, reference Figure 7 As shown, the step of inputting the audio segment to be recognized into the second recognition model to obtain a second recognition result of the audio segment to be recognized includes:

[0121] Step S710: input the audio segment to be identified into the second feature extractor to obtain the hidden features of the audio segment to be identified;

[0122] Step S720: input the hidden layer features into the second classifier to obtain a prediction score corresponding to the audio segment to be identified;

[0123] Step S730. Obtain a second prediction score of the audio segment to be recognized based on the prediction score of the audio segment to be recognized, and obtain the second recognition result based on the second prediction score.

[0124] Below, step S710-step S730 will be further explained and illustrated. Specifically, the audio segment to be identified of the audio to be identified is input into the second feature extractor to obtain the hidden features of the audio segment to be identified. The second feature extractor can be a Mert pre-trained model. Since the Mert model is a model based on the Transformer structure, each of its hidden layers contains different audio representation information. The shallow layer reflects the audio signal features, and the deep layer reflects the audio abstract features. The difference between AI audio and real audio may appear at each level. Therefore, the data of its 13 hidden layers can be used as features. After extracting the hidden features of the audio segment to be identified, the extracted hidden features can be input into the second classifier to obtain the prediction score corresponding to the audio segment to be identified, and the second prediction score of the audio segment to be identified is obtained according to the prediction score of the audio segment to be identified. After obtaining the second prediction score of the audio segment to be identified, the second recognition result of the audio to be identified can be obtained according to the second prediction score.

[0125] refer to Figure 8 As shown, obtaining the second recognition result according to the second prediction score includes:

[0126] Step S810: when the second prediction score is greater than a third preset value, determining that the audio to be identified is a first type of audio;

[0127] Step S820. When the second prediction score is less than the fourth preset value, determine that the audio to be recognized is a second type of audio;

[0128] Step S830. When the second prediction score is not less than the fourth preset value and not greater than the third preset value, obtain a third prediction score of the audio to be recognized according to the first prediction score and the second prediction score of the audio to be recognized;

[0129] Step S840. Determine the type of the audio to be recognized according to the third prediction score of the audio to be recognized.

[0130] Next, steps S810 - S840 will be further explained and described. Specifically, when the second prediction score of the audio to be recognized is greater than the third preset value, determine that the audio to be recognized is an AI audio; when the second prediction score of the audio to be recognized is less than the fourth preset value, determine that the audio to be recognized is a non - AI audio; when the second prediction score of the audio to be recognized is not less than the fourth preset value and not greater than the third preset value, obtain a third prediction score of the audio to be recognized according to the first prediction score and the second prediction score of the audio to be recognized, and determine the type of the audio to be recognized according to the third prediction score. Among them, the third preset value can be a2, the fourth preset value can be b2, b2 < a2, a2 can be 0.6, b2 can be 0.4, and the values of a2 and b2 are not specifically limited in this exemplary embodiment; the first prediction score and the second prediction score of the audio to be recognized can be averaged to obtain the third prediction score.

[0131] Further, referring to Fig. 9 as shown, the determining the type of the audio to be recognized according to the third prediction score of the audio to be recognized includes:

[0132] Step S910. When the third prediction score is greater than the fifth prediction value, determine that the audio to be recognized is a first type of audio;

[0133] Step S920. When the third prediction score is not greater than the fifth prediction value, determine that the audio to be recognized is a second type of audio.

[0134] Next, steps S910 and S920 will be further explained and described. Specifically, when the third prediction score is greater than the fifth prediction value, determine that the audio to be recognized is an AI audio; when the third prediction score of the audio to be recognized is not greater than the fifth prediction value, determine that the audio to be recognized is a non - AI audio.

[0135] Exemplary Devices

[0136] After introducing the audio recognition method of the exemplary embodiments of the present disclosure, next, referring to Fig.10 A data communication device according to an exemplary embodiment of the present disclosure is described.

[0137] refer to Fig.10 As shown, the audio recognition device of the exemplary embodiment of the present disclosure may include: an audio processing module 1010, a first recognition module 1020 and a second recognition module 1030; wherein:

[0138] The audio processing module 1010 is used to obtain the audio to be recognized, pre-process the audio to be recognized, and obtain an audio segment to be recognized corresponding to the audio to be recognized;

[0139] A first recognition module 1020 is used to obtain a first recognition model, input the audio segment to be recognized into the first recognition model, and obtain a first recognition result;

[0140] The second recognition module 1030 is used to obtain a second recognition model when the first recognition result is a preset result, input the audio segment to be recognized into the second recognition model, and obtain a second recognition result of the audio segment to be recognized.

[0141] According to an exemplary embodiment of the present disclosure, the audio processing module includes:

[0142] A format conversion module, used for converting the audio to be recognized into a target format, and segmenting the audio to be recognized in the target format to obtain a first audio segment;

[0143] The silence detection module is used to perform silence detection on the first audio segment to obtain the silence segment included in the first audio segment, and delete the silence segment to obtain the audio segment to be identified.

[0144] According to an exemplary embodiment of the present disclosure, the first identification module includes:

[0145] A first recognition model generating module, used for acquiring a first feature extractor and a first classifier, and forming the first recognition model based on the first feature extractor and the first classifier;

[0146] A first loss function determination module, used to obtain sample audio, train the first recognition model using the sample audio, obtain a first output result of the first recognition model, and determine a first loss function of the first recognition model using the first output result;

[0147] The first recognition model updating module is used to update the parameters of the first recognition model based on the first loss function.

[0148] According to an exemplary embodiment of the present disclosure, the first identification module includes:

[0149] A first prediction module, configured to input the audio segment to be recognized into the first recognition model to obtain a prediction score corresponding to the audio segment to be recognized;

[0150] A first prediction score determination module, configured to obtain a first prediction score of the audio to be identified according to the prediction score of the audio segment to be identified, and determine that the audio to be identified is a first type of audio when the first prediction score is greater than a first preset value;

[0151] A second audio type determination module, configured to determine that the audio to be identified is a second type of audio when the first prediction score is less than a second preset value;

[0152] The third type audio determination module is used to determine that the audio to be identified is the third type audio when the first prediction score is not less than the second preset value and not greater than the first preset value.

[0153] According to an exemplary embodiment of the present disclosure, the second identification module includes:

[0154] A second recognition model determination module, used in the second recognition model acquisition module, when the audio to be recognized is the third type of audio, acquires the second recognition model;

[0155] The second recognition result acquisition module is used to input the audio segment to be recognized of the audio to be recognized into the second recognition model to obtain the second recognition result.

[0156] According to an exemplary embodiment of the present disclosure, the second recognition model determination module includes:

[0157] A second recognition model determination module, used for obtaining a second feature extractor and a second classifier, and forming the second recognition model based on the second feature extractor and the second classifier;

[0158] A second output result acquisition module is used to acquire a sample audio, input the sample audio into the second feature extractor, obtain a hidden layer feature of the sample audio, input the hidden layer feature into the second classifier, and obtain a second output result of the sample audio;

[0159] The second recognition model updating module is used to determine a second loss function according to the second output result, and update the parameters of the second classifier based on the second loss function.

[0160] According to an exemplary embodiment of the present disclosure, the second recognition result acquisition module includes:

[0161] A feature extraction module, used for inputting the audio segment to be identified into the second feature extractor to obtain the hidden layer features of the audio segment to be identified;

[0162] A score prediction module, used for inputting the hidden layer features into the second classifier to obtain a prediction score corresponding to the audio segment to be identified;

[0163] The result acquisition module is used to obtain a second prediction score of the audio to be recognized according to the prediction score of the audio segment to be recognized, and obtain the second recognition result according to the second prediction score.

[0164] According to an exemplary embodiment of the present disclosure, the result acquisition module includes:

[0165] A first type determination module, configured to determine that the audio to be identified is a first type of audio when the second prediction score is greater than a third preset value;

[0166] A second type determination module, configured to determine that the audio to be identified is a second type of audio when the second prediction score is less than a fourth preset value;

[0167] A third prediction score determination module, configured to obtain a third prediction score of the audio to be identified according to the first prediction score of the audio to be identified and the second prediction score when the second prediction score is not less than the fourth preset value and not greater than the third preset value;

[0168] The third prediction score identification module is used to determine the type of the audio to be identified according to the third prediction score of the audio to be identified.

[0169] According to an exemplary embodiment of the present disclosure, the third prediction score identification module includes:

[0170] A first recognition module, configured to determine that the audio to be recognized is a first type of audio when the third prediction score is greater than a fifth prediction value;

[0171] The second recognition module is used to determine that the audio to be recognized is a second type of audio when the third prediction score is not greater than the fifth prediction value.

[0172] Since the functional modules of the audio recognition device in the disclosed embodiment are the same as those in the disclosed embodiment of the audio recognition method, they will not be described in detail here.

[0173] Exemplary Storage Media

[0174] After introducing the audio recognition method and device according to the exemplary embodiment of the present disclosure, Fig.11A computer-readable storage medium according to an exemplary embodiment of the present disclosure is described.

[0175] refer to Fig.11 As shown, a program product 1100 for implementing the above method according to an embodiment of the present disclosure is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0176] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0177] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0178] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0179] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user computing device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0180] Exemplary Electronic Devices

[0181] After introducing the storage medium of the exemplary embodiment of the present disclosure, next, reference is made to Fig. 9 An electronic device according to an exemplary embodiment of the present disclosure is described.

[0182] Fig.12 The electronic device 1200 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0183] like Fig.12 As shown, the electronic device 1200 is in the form of a general computing device. The components of the electronic device 1200 may include, but are not limited to: the at least one processing unit 1210, the at least one storage unit 1220, a bus 1230 connecting different system components (including the storage unit 1220 and the processing unit 1210), and a display unit 1240.

[0184] The storage unit stores a program code, which can be executed by the processing unit 1210, so that the processing unit 1210 performs the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of this specification. For example, the processing unit 1210 can perform the following steps: Figure 1 Steps S110 to S130 shown in FIG.

[0185] The storage unit 1220 may include a volatile storage unit, such as a random access storage unit (RAM) 12201 and / or a cache storage unit 12202 , and may further include a read-only storage unit (ROM) 12203 .

[0186] The storage unit 1220 may also include a program / utility 12204 having a set (at least one) of program modules 12205, such program modules 12205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0187] The bus 1230 may include a data bus, an address bus, and a control bus.

[0188] The electronic device 1200 may also communicate with one or more external devices 1300 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), and such communication may be performed through an input / output (I / O) interface 1250. The electronic device 1200 also includes a display unit 1240, which is connected to the input / output (I / O) interface 1250 for display. In addition, the electronic device 1200 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter 1260. As shown, the network adapter 1260 communicates with other modules of the electronic device 1200 through a bus 1230. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0189] It should be noted that although the song relationship identification device and several modules or sub-modules of the song relationship identification device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units / modules described above can be concretized in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided to be concretized by multiple units / modules.

[0190] In addition, although the operations of the disclosed method are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0191] Although the spirit and principle of the present disclosure have been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present disclosure is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims.

Claims

1. An audio recognition method, characterized in that: include: Acquire the audio to be recognized, and preprocess the audio to be recognized to obtain an audio segment to be recognized corresponding to the audio to be recognized; Acquire a first recognition model, input the audio segment to be recognized into the first recognition model, and obtain a first recognition result; When the first recognition result is a preset result, a second recognition model is obtained, and the audio segment to be recognized is input into the second recognition model to obtain a second recognition result of the audio segment to be recognized.

2. The method according to claim 1, characterized in that The step of inputting the audio segment to be recognized into the first recognition model to obtain a first recognition result includes: Inputting the audio segment to be recognized into the first recognition model to obtain a prediction score corresponding to the audio segment to be recognized; Obtaining a first prediction score of the audio to be identified according to the prediction score of the audio segment to be identified, and determining that the audio to be identified is a first type of audio when the first prediction score is greater than a first preset value; When the first prediction score is less than a second preset value, determining that the audio to be identified is a second type of audio; When the first prediction score is not less than the second preset value and not greater than the first preset value, it is determined that the audio to be identified is the third type of audio.

3. The method according to claim 2, characterized in that When the first recognition result is a preset result, obtaining a second recognition model, inputting the audio segment to be recognized into the second recognition model, and obtaining a second recognition result of the audio segment to be recognized, includes: When the audio to be recognized is the third type of audio, obtaining the second recognition model; The audio segment to be recognized of the audio to be recognized is input into the second recognition model to obtain the second recognition result.

4. The method according to claim 3, characterized in that Before acquiring the second recognition model, the method further includes: Acquire a second feature extractor and a second classifier, and construct the second recognition model based on the second feature extractor and the second classifier; Acquire sample audio, input the sample audio into the second feature extractor to obtain hidden features of the sample audio, input the hidden features into the second classifier to obtain a second output result of the sample audio; A second loss function is determined according to the second output result, and parameters of the second classifier are updated based on the second loss function.

5. The method according to claim 4, characterized in that The step of inputting the audio segment to be recognized into the second recognition model to obtain a second recognition result of the audio segment to be recognized includes: Inputting the audio segment to be identified into the second feature extractor to obtain the hidden features of the audio segment to be identified; Inputting the hidden layer features into the second classifier to obtain a prediction score corresponding to the audio segment to be identified; A second prediction score of the audio segment to be recognized is obtained according to the prediction score of the audio segment to be recognized, and the second recognition result is obtained according to the second prediction score.

6. The method according to claim 5, characterized in that The obtaining the second recognition result according to the second prediction score includes: When the second prediction score is greater than a third preset value, determining that the audio to be identified is a first type of audio; When the second prediction score is less than a fourth preset value, determining that the audio to be identified is a second type of audio; When the second prediction score is not less than the fourth preset value and not greater than the third preset value, obtaining a third prediction score of the audio to be recognized according to the first prediction score of the audio to be recognized and the second prediction score; The type of the audio to be recognized is determined according to the third prediction score of the audio to be recognized.

7. The method according to claim 6, characterized in that The determining the type of the audio to be recognized according to the third prediction score of the audio to be recognized comprises: When the third prediction score is greater than the fifth prediction value, determining that the audio to be identified is the first type of audio; When the third prediction score is not greater than the fifth prediction value, it is determined that the audio to be identified is the second type of audio.

8. An audio recognition device, characterized in that: include: An audio processing module, used to obtain the audio to be recognized, pre-process the audio to be recognized, and obtain an audio segment to be recognized corresponding to the audio to be recognized; A first recognition module, configured to obtain a first recognition model, input the audio segment to be recognized into the first recognition model, and obtain a first recognition result; The second recognition module is used to obtain a second recognition model when the first recognition result is a preset result, input the audio segment to be recognized into the second recognition model, and obtain a second recognition result of the audio segment to be recognized.

9. A computer program product, comprising a computer program product, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: processor; and a memory for storing executable instructions for the processor; The processor is configured to perform the method of any one of claims 1 to 7 by executing the executable instructions.

Images