Audio recognition method and device and electronic equipment

Abstract

The invention provides an audio recognition method and device and electronic equipment. The method comprises the steps of obtaining to-be-recognized audio; determining at least one first audio clip and at least one second audio clip from the to-be-recognized audio through a pre-trained first recognition model; the ratio of the song duration in the first audio clip meets a first specified condition, and the ratio of the song duration in the second audio clip meets a second specified condition; determining a first probability that the first audio clip belongs to the AI flipping audio and a second probability that the second audio clip belongs to the AI generated audio through a second identification model; determining an audio recognition result of the to-be-recognized audio according to the first probability and the second probability; the audio recognition result is AI audio turning and singing, AI generated audio or normal audio. According to the method, audio recognition is carried out on the singing sound segments and the non-singing sound segments, misleading of the non-singing sound segments to the recognition result is avoided, the recognition accuracy of the singing sound segments is improved, and then the recognition accuracy of the whole audio is improved.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to an audio recognition method, apparatus and electronic device. Background Technology

[0002] With the rapid development of AI (Artificial Intelligence) technology, especially the breakthroughs in speech synthesis and music generation, AI-generated content is becoming increasingly common in the music industry. The emergence of AI-generated songs and AI-covered songs presents new challenges to copyright protection and content management. Existing AI song recognition models, when identifying AI-covered songs, may be misled by the "real" accompaniment, leading to insufficient sensitivity in recognizing AI traces in the vocal parts and a significant decrease in overall recognition rate. Summary of the Invention

[0003] In view of this, the purpose of this disclosure is to provide an audio recognition method, apparatus, and electronic device that determines the recognition result of the audio by identifying the singing segment and the non-singing segment of the audio to be recognized, the probability of identifying the singing segment as an AI cover audio, and the probability of identifying the non-singing segment as an AI generated audio. This method avoids the non-singing segment from misleading the recognition result and improves the recognition accuracy of the singing segment, thereby improving the overall audio recognition accuracy.

[0004] In a first aspect, embodiments of this disclosure provide an audio recognition method, the method comprising: acquiring an audio to be recognized; determining at least one first audio segment and at least one second audio segment from the audio to be recognized using a pre-trained first recognition model; wherein the proportion of singing duration in the first audio segment satisfies a first specified condition, and the proportion of singing duration in the second audio segment satisfies a second specified condition; inputting the first audio segment and the second audio segment into the pre-trained second recognition model, and determining a first probability of the first audio segment and a second probability of the second audio segment using the second recognition model; the first probability being the probability that the first audio segment belongs to an AI cover song, and the second probability being the probability that the second audio segment belongs to an AI-generated audio; determining an audio recognition result of the audio to be recognized based on the first probability and the second probability; wherein the audio recognition result is: an AI cover song, an AI-generated audio, or normal audio.

[0005] Secondly, embodiments of this disclosure provide an audio recognition device, comprising: a first recognition module, configured to acquire audio to be recognized; and determine at least one first audio segment and at least one second audio segment from the audio to be recognized using a pre-trained first recognition model; wherein the proportion of singing duration in the first audio segment satisfies a first specified condition, and the proportion of singing duration in the second audio segment satisfies a second specified condition; a second recognition module, configured to input the first audio segment and the second audio segment into the pre-trained second recognition model, and determine a first probability of the first audio segment and a second probability of the second audio segment using the second recognition model; the first probability is the probability that the first audio segment belongs to AI cover audio, and the second probability is the probability that the second audio segment belongs to AI generated audio; and a result determination module, configured to determine the audio recognition result of the audio to be recognized based on the first probability and the second probability; wherein the audio recognition result is: AI cover audio, AI generated audio, or normal audio.

[0006] Thirdly, embodiments of this disclosure provide an electronic device, including a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the audio recognition method of any of the first aspects.

[0007] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the audio recognition method of any one of the first aspects.

[0008] The embodiments disclosed herein bring the following beneficial effects: This disclosure provides an audio recognition method, apparatus, and electronic device, which acquires audio to be recognized; determines at least one first audio segment and at least one second audio segment from the audio to be recognized using a pre-trained first recognition model; wherein the proportion of singing duration in the first audio segment satisfies a first specified condition, and the proportion of singing duration in the second audio segment satisfies a second specified condition; inputs the first audio segment and the second audio segment into the pre-trained second recognition model, and determines a first probability of the first audio segment and a second probability of the second audio segment using the second recognition model; the first probability is the probability that the first audio segment belongs to AI cover audio, and the second probability is the probability that the second audio segment belongs to AI generated audio; and determines the audio recognition result of the audio to be recognized based on the first probability and the second probability; wherein the audio recognition result is: AI cover audio, AI generated audio, or normal audio. In this method, by determining the singing segment and the non-singing segment of the audio to be identified, the probability of identifying the singing segment as an AI cover audio and the probability of identifying the non-singing segment as an AI-generated audio are determined, and the recognition result of the audio to be identified is determined. By identifying the singing segment and the non-singing segment separately, this method avoids the non-singing segment from misleading the recognition result, improves the recognition accuracy of the singing segment, and thus improves the overall audio recognition accuracy.

[0009] Other features and advantages of this disclosure will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the disclosure. The objects and other advantages of this disclosure are realized and obtained through the structures particularly pointed out in the description, claims and drawings.

[0010] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the specific embodiments of this disclosure or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0012] Figure 1 A flowchart of an audio recognition method provided in this disclosure embodiment; Figure 2 A flowchart illustrating an audio recognition method provided in an embodiment of this disclosure; Figure 3 A schematic diagram of model training for an audio recognition method provided in this embodiment of the present disclosure; Figure 4A schematic diagram of model training for another audio recognition method provided in an embodiment of this disclosure; Figure 5 A schematic diagram of model training for another audio recognition method provided in an embodiment of this disclosure; Figure 6 This is a schematic diagram of the structure of an audio recognition device provided in an embodiment of the present disclosure; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0014] With the rapid development of AI (Artificial Intelligence) technology, especially the breakthroughs in speech synthesis and music generation, AI-generated content is becoming increasingly common in the music industry. The emergence of AI-generated songs and AI-covered songs has brought new challenges to copyright protection and content management. To address these challenges, existing technologies mainly focus on the following two aspects for identification and detection: On the one hand, identifying AI-generated songs primarily relies on analyzing the overall characteristics of the song. AI-generated songs are typically created from scratch by deep learning models such as Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), or Transformers. Their melody, harmony, rhythm, orchestration, and even the vocals may originate from AI creation. Therefore, methods for identifying these songs often focus on capturing the differences between AI-generated songs and human-composed songs in terms of musical structure, compositional patterns, and timbre details. For example, some technical solutions analyze the macroscopic structure of the music, such as the repetition of musical phrases, the novelty of chord progressions, and the overall "artificial traces." Furthermore, by analyzing the spectral characteristics, temporal characteristics, and higher-order statistical features of the audio, classifiers can be trained to distinguish between AI-generated songs and human-composed songs.

[0015] On the other hand, existing technologies also focus extensively on the recognition of AI-synthesized speech or singing. This mainly involves technologies such as text-to-speech (TTS) synthesis and voice conversion. The focus of this type of recognition is to detect whether there are traces of artificial synthesis in the audio, rather than the original source of the music. For example, by analyzing the acoustic features of the speech, such as the smoothness of the fundamental frequency, the distribution of formants, the naturalness of the spectral envelope, and the presence of specific synthesis artifacts, it is possible to determine whether the speech is synthesized. Deep learning models, especially convolutional neural networks (CNNs), are widely used to extract these subtle acoustic features and train classifiers to distinguish between real and fake speech.

[0016] While existing technologies have made some progress in AI-generated song recognition and AI-synthesized speech / voice recognition, they exhibit significant limitations when faced with the specific scenario of "AI-generated song covers," mainly in the following two aspects: First, the recognition rate is low: Existing recognition models for AI-generated songs, because their training data and recognition targets are mainly music entirely generated by AI, tend to capture the overall AI generation pattern of the music. When faced with AI-generated covers, because the accompaniment is "real," these models may be misled by the authenticity of the accompaniment, resulting in insufficient sensitivity to the AI ​​traces in the vocal parts, thus significantly reducing the overall recognition rate. They cannot effectively distinguish between "AI-created music" and "music with AI-replaced vocals."

[0017] Secondly, it can easily lead to AI-generated songs being misclassified as AI covers: Existing recognition models for AI-synthesized speech or singing, while capable of detecting AI synthesis traces in the singing, typically lack the ability to understand the overall musical structure. When faced with a song entirely generated by AI, these models tend to only recognize the AI ​​synthesis characteristics of the singing, ignoring the fact that the accompaniment is also AI-generated, thus incorrectly classifying it as an "AI cover." This misclassification not only affects the accuracy of recognition but can also lead to confusion in the classification of AI-generated content, causing problems for copyright management and content distribution.

[0018] In summary, the emergence of AI-generated cover songs exposes a significant vulnerability in existing technology: how to effectively distinguish between "AI-generated music" and "music with AI-replaced vocals." Since the accompaniment in AI-generated cover songs is not AI-generated, traditional AI song recognition methods are ineffective. Furthermore, relying solely on AI synthesis traces in the vocals cannot comprehensively determine the AI ​​attributes of the song. This complexity necessitates a more refined, multi-dimensional recognition method capable of comprehensively considering all components of a song and specifically identifying the unique characteristics of AI-generated cover songs. Currently, streaming platforms urgently need a technical solution that can significantly improve the recognition rate of AI-generated cover songs and reduce false positives to address the legal risks and copyright issues arising from the increasing dissemination of unauthorized AI-generated cover songs. Based on this, this disclosure provides an audio recognition method, apparatus, and electronic device, which can be applied to audio recognition technology scenarios.

[0019] To facilitate understanding of this embodiment, a detailed description of an audio recognition method disclosed in this disclosure will be provided first, such as... Figure 1 As shown, the method includes the following steps: Step S102: Obtain the audio to be identified; determine at least one first audio segment and at least one second audio segment from the audio to be identified using a pre-trained first recognition model; wherein the proportion of singing duration in the first audio segment satisfies a first specified condition, and the proportion of singing duration in the second audio segment satisfies a second specified condition. The audio to be identified can be a complete song or a portion of a complete piece of music, or it can be a dialogue with background music, a crosstalk performance with background music, or a monologue with background music, etc. The first recognition model that has been pre-trained is usually called a singing recognition model, and it can be a deep learning sequence model, such as a Long Short-Term Memory (LSTM) network, a gated recurrent unit (GRU), or a Transformer, etc.

[0020] The aforementioned percentage of singing duration refers to the ratio of singing duration to the total duration of the segment. The first specified condition can be that the percentage of singing duration is greater than a first value, and the second specified condition can be that the percentage of singing duration is less than a second value. The first value can be 0.5, 0.6, 0.7, 0.8, etc., and the second value can be 0.1, 0.2, 0.3, etc. The first and second values ​​can be preset as needed.

[0021] In one possible approach, the audio to be identified is input into a first recognition model, which segments the audio to be identified into multiple audio segments; wherein each audio segment has the same duration; and then the multiple audio segments are divided into at least one first audio segment and at least one second audio segment.

[0022] In another possible approach, the audio to be identified is sliced ​​to obtain multiple audio segments, each of which has the same duration; the audio segments are then input into a first recognition model, which determines at least one first audio segment and at least one second audio segment from the multiple audio segments.

[0023] Specifically, it can identify whether there is singing in an audio segment. If so, the audio segment is designated as the first audio segment; otherwise, it is designated as the second audio segment. Alternatively, it can identify the ratio of the singing duration in an audio segment to the total segment length. If the ratio is greater than or equal to a first value, the audio segment is designated as the first audio segment; if the ratio is less than or equal to a second value, it is designated as the second audio segment. If the ratio is less than the first value but greater than the second value, the audio segment is deleted.

[0024] Step S104: Input the first audio segment and the second audio segment into the pre-trained second recognition model, and determine the first probability of the first audio segment and the second probability of the second audio segment through the second recognition model; the first probability is the probability that the first audio segment belongs to the AI ​​cover audio, and the second probability is the probability that the second audio segment belongs to the AI ​​generated audio. Optionally, the second recognition model described above can be a model structure with a single classifier, or a model structure with an audio track segmentation network and a classifier connected in series. The classifier can be a neural network classifier, such as a residual network, a gated recurrent unit, or even a graph neural network or AASIST (Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks). The audio track segmentation network can be U-net (Convolutional Networks for Biomedical Image Segmentation), FCN (Fully Convolutional Network), etc.

[0025] In one possible approach, a first audio segment is input into a pre-trained second recognition model, which outputs the probability that the first audio segment belongs to an AI-generated cover audio and the probability that it belongs to an AI-generated audio. At the same time, a second audio segment is input into the pre-trained second recognition model, which outputs the probability that the second audio segment belongs to an AI-generated cover audio and the probability that it belongs to an AI-generated audio.

[0026] In another possible approach, the second recognition model includes: an AI cover song recognition model and an AI generation recognition model; a first audio segment is input into the AI ​​cover song recognition model, and a first probability of the first audio segment is determined by the AI ​​cover song recognition model; a second audio segment is input into the AI ​​generation recognition model, and a second probability of the second audio segment is determined by the AI ​​generation recognition model.

[0027] Step S106: Determine the audio recognition result of the audio to be recognized based on the first probability and the second probability; wherein, the audio recognition result is: AI cover audio, AI generated audio, or normal audio.

[0028] The aforementioned AI-generated cover audio refers to background music created by a human artist, while the vocals are generated by AI. The aforementioned AI-generated audio refers to both background music and vocals being generated by AI. The aforementioned normal audio typically refers to human-created audio, i.e., music composed by a human artist, and vocals sung by a human artist.

[0029] In one approach, a first mean is calculated from the average of the first probabilities, and a second mean is obtained from the average of the second probabilities. If the first mean is greater than the second mean and the first mean is greater than a first mean threshold, and the second mean is less than the second mean threshold, the audio to be identified is determined to be an AI cover audio. If the first mean is less than the second mean and the first mean is less than the second mean threshold, and the second mean is greater than the first mean threshold, the audio to be identified is determined to be AI-generated audio; otherwise, the audio to be identified is determined to be normal audio.

[0030] In another approach, a first mean is calculated from the average of the first probabilities, and a second mean is obtained from the average of the second probabilities. If the first mean is greater than a first preset value and the second mean is less than a second preset value, the audio to be identified is determined to be an AI cover audio. If the first mean is greater than the first preset value and the second mean is greater than the first preset value, the audio to be identified is determined to be AI generated audio. If the first mean is less than the second preset value and the second mean is less than the second preset value, the audio to be identified is determined to be normal audio.

[0031] This disclosure provides an audio recognition method, which involves acquiring an audio file to be recognized; determining at least one first audio segment and at least one second audio segment from the audio file using a pre-trained first recognition model; wherein the proportion of singing duration in the first audio segment satisfies a first specified condition, and the proportion of singing duration in the second audio segment satisfies a second specified condition; inputting the first and second audio segments into the pre-trained second recognition model, and determining a first probability of the first audio segment and a second probability of the second audio segment using the second recognition model; the first probability is the probability that the first audio segment belongs to an AI cover song, and the second probability is the probability that the second audio segment belongs to an AI-generated audio file; and determining the audio recognition result of the audio file based on the first and second probabilities; wherein the audio recognition result is: an AI cover song, an AI-generated audio file, or normal audio. In this method, by determining the singing segment and the non-singing segment of the audio to be identified, the probability of identifying the singing segment as an AI cover audio and the probability of identifying the non-singing segment as an AI-generated audio are determined, and the recognition result of the audio to be identified is determined. By identifying the singing segment and the non-singing segment separately, this method avoids the non-singing segment from misleading the recognition result, improves the recognition accuracy of the singing segment, and thus improves the overall audio recognition accuracy.

[0032] One possible implementation of the above-mentioned step of determining at least one first audio segment and at least one second audio segment from the audio to be identified by a pre-trained first recognition model is as follows: the audio to be identified is processed into audio segments to obtain multiple audio segments; the multiple audio segments are input into the first recognition model, and at least one first audio segment and at least one second audio segment are determined from the multiple audio segments by the first recognition model.

[0033] Each audio segment has the same duration. Optionally, the audio to be identified can be sliced ​​into multiple audio segments of fixed duration. These multiple audio segments are then input into a first recognition model. The first recognition model determines the ratio of the singing duration to the segment duration in each audio segment, obtaining the percentage of singing duration. If the percentage of singing duration is greater than a first value, the audio segment is identified as the first audio segment; if the percentage is less than a second value, the audio segment is identified as the second audio segment. Otherwise, the audio segment is deleted.

[0034] In this embodiment, the first recognition model is a model structure consisting of a feature extraction network and a GRU connected in series. The feature extraction network is used to extract features of audio segments, such as Mel spectrum, MFCC (Mel frequency cepstral coefficients), or other acoustic features. In this embodiment, the feature extraction network is used to extract Mel spectrum features.

[0035] For example, such as Figure 2As shown, the song audio (i.e. the audio to be identified mentioned above) is obtained, sliced, and then input into the singing voice recognition model (i.e. the first recognition model mentioned above), and the singing voice segment (i.e. the first audio segment mentioned above) and the non-singing voice segment (i.e. the second audio segment mentioned above) are output.

[0036] The aforementioned second recognition model includes: an AI cover song recognition model and an AI-generated recognition model; One possible implementation of the above-described steps, which involve inputting the first and second audio segments into a pre-trained second recognition model and determining the first probability of the first audio segment and the second probability of the second audio segment using the second recognition model, is as follows: (1) Input the first audio segment into the AI ​​cover song recognition model, and determine the first probability of the first audio segment through the AI ​​cover song recognition model; The aforementioned AI cover song recognition model is a model structure that connects an audio track segmentation network and a classifier network. In this embodiment, the audio track segmentation network can be a Unet network, and the classifier network can be an AASIST graph neural network. The AI ​​cover song recognition model calculates the probability that each first audio segment belongs to an AI cover audio. Evaluating whether the vocal part of the song is an AI cover is the key step in directly determining whether a song is an AI cover.

[0037] (2) Input the second audio segment into the AI ​​generation and recognition model, and determine the second probability of the second audio segment through the AI ​​generation and recognition model.

[0038] The aforementioned AI-generated recognition model comprises two parts: a feature extractor and a classifier. The feature extractor is a neural network feature extractor, such as RawNet. The classifier is a neural network classifier, such as a residual network, gated recurrent unit, graph neural network classifier, or AASIST graph neural network. This AI-generated recognition model calculates the probability that each second audio segment belongs to AI-generated audio. It evaluates whether the non-vocal parts of the song were generated by AI. If the probability of non-vocal segments being identified as AI songs is high, the overall likelihood of the song being an AI song increases, which helps distinguish between AI-generated audio and AI-covered audio.

[0039] By breaking down a song into vocal and non-vocal segments and analyzing these two types of segments using different AI recognition models in parallel, this divide-and-conquer strategy not only improves recognition accuracy but also reduces computational latency.

[0040] One possible implementation of the above steps for determining the audio recognition result of the audio to be recognized based on the first probability and the second probability is as follows: (1) Calculate the first average of the first probability of at least one first audio segment, and calculate the second average of the second probability of at least one second audio segment; The first average value is denoted as cover_p, and the second average value is denoted as ai_p.

[0041] (2) Determine the audio recognition result of the audio to be recognized according to the first average value, the second average value, and the preset conditions.

[0042] Optionally, the above preset conditions include the first preset condition, the second preset condition, and the third preset condition; if the first average value and the second average value meet the first preset condition, determine that the audio recognition result of the audio to be recognized is an AI cover audio; if the first average value and the second average value meet the second preset condition, determine that the audio recognition result of the audio to be recognized is an AI-generated audio; if the first average value and the second average value meet the third preset condition, determine that the audio recognition result of the audio to be recognized is a normal audio.

[0043] Specifically, if the first average value is greater than or equal to the first threshold (T1) and the second average value is less than or equal to the second threshold (T2), determine that the audio recognition result of the audio to be recognized is an AI cover audio; if the first average value is greater than or equal to the first threshold and the second average value is greater than or equal to the first threshold, determine that the audio recognition result of the audio to be recognized is an AI-generated audio; if the first average value is less than or equal to the second threshold and the second average value is less than the second threshold, determine that the audio recognition result of the audio to be recognized is a normal audio.

[0044] For example, if cover_p ≥ T1 and ai_p ≤ T2, it is determined as an AI cover audio; if cover_p ≥ T1 and ai_p ≥ T1, it indicates that both the singing and non-singing parts of the song may be AI-generated, and it is determined as an AI-generated audio; if cover_p < T2 and ai_p < T2, it is determined as a normal audio.

[0045] Based on the weighted or logical judgment of the AI cover probability of the singing segment and the AI generation probability of the non-singing segment, the conclusion of whether the song is an AI cover is finally obtained. This comprehensive judgment avoids misjudgments that may be caused by a single model, especially distinguishing between AI-generated audio and AI cover audio.

[0046] The above method further includes: if the first average value and the second average value do not meet the preset conditions, determine that the recognition result of the audio to be recognized is the target recognition result, and the target recognition result indicates that the audio to be recognized needs to be further recognized based on auxiliary features or requires manual recognition.

[0047] If the above preset conditions are not met, it indicates that there is a relatively high AI probability in the non-singing part, and it may be that AI cover features are mixed in the AI-generated audio. At this time, more context features (such as external data like song copyright information, singer information, etc.) need to be combined for further auxiliary judgment, or it is marked as a suspected AI cover audio for manual review.

[0048] The above method includes: acquiring multiple first training samples, each first training sample carrying a first sample label, the first sample label including a first tag and a second tag, the first tag indicating that the first training sample is a singing segment, and the second tag indicating that the first training sample is a non-singing segment; inputting the first training samples into a first recognition model to be trained, and determining the sample classification result of the first training samples through the first recognition model; calculating a first loss function value based on the sample classification result of the first training samples and the first sample label of the first training samples; and adjusting the model parameters of the first recognition model based on the first loss function value until the first loss function value meets a first preset condition, thereby obtaining a trained first recognition model.

[0049] Optionally, the aforementioned multiple first training samples are determined as follows: Multiple songs are acquired, each carrying a vocal position label, such as a lyrics file with timestamps and word-by-word lyrics. These files cover the locations where vocal segments appear. Data sources can include songs of various music genres, languages, and vocal characteristics, as well as instrumental music, background noise, and other non-vocal audio. Each song is divided into multiple short audio segments of equal length (e.g., 4 seconds in length). Based on the timestamps of the lyrics file (usually accurate to milliseconds), the proportion of vocal duration in each short audio segment can be calculated. If the proportion of vocal duration is greater than 0.8, it is considered a vocal segment, and the first sample label is set as the first label; if it is less than 0.2, it is considered a non-vocal segment, and the first sample label is set as the second label. All other cases are discarded.

[0050] Example: timestamp lyrics file with lyrics word by word: [12210,6860](12210,890,0)After (13100,1150,0)Come (14250,410,0)I (14660,370,0)Always (15030,380,0)Calculate (15410,460,0)Learn (15870,420,0)Learn (16290,970,0)Like (17880,370,0)How (18250,340,0)Go (18590,480,0)Love.

[0051] The first recognition model to be trained can be a deep learning-based sequence model, such as a Long Short-Term Memory (LSTM) network, a Gated Recurrent Unit (GRU), or a Transformer. Such models can effectively capture temporal features in audio sequences. Audio feature extraction uses commonly used features in speech recognition, such as Mel spectrum, MFCC (Mel frequency cepstral coefficients), or other acoustic features.

[0052] In this implementation, Mel-spectral features are used, with GRU as the model, and the output is a binary classification result to indicate whether the segment contains singing.

[0053] For example, such as Figure 3 As shown, the audio and vocal location information of the song are obtained. First, the audio is segmented, then audio features are extracted, and then the features of each audio segment are input into the GRU. The output is the sample classification result of each first training sample.

[0054] Optionally, supervised learning can be used for training. The loss function chosen is binary cross-entropy loss, and optimizers such as Adam and SGD can be used. Data augmentation techniques, such as pitch shifting, time stretching, and adding noise, are employed during training to improve the model's generalization ability.

[0055] The above method includes: acquiring multiple second training samples, each carrying a second sample label, which includes a third label and a fourth label. The third label indicates that the second training sample is AI-generated audio, and the fourth label indicates that the second training sample is normal audio. The second training samples are input into an AI generation and recognition model to be trained, and the AI ​​generation and recognition model determines the sample recognition result of the second training samples. Based on the sample classification result of the second training samples and the second sample label of the second training samples, a second loss function value is calculated. Based on the second loss function value, the model parameters of the AI ​​generation and recognition model are adjusted until the second loss function value meets a second preset condition, thereby obtaining a trained AI generation and recognition model.

[0056] The second training sample was determined as follows: a large amount of normal audio (human-composed songs) and AI-generated song data were collected. The AI-generated song data should cover various styles and types of music produced by current mainstream AI music generation technologies. The normal audio data needed to be broadly representative to ensure the model could learn the authentic characteristics of human-composed music. This data also needed to be fragmented (to maintain consistency with the fragments in the song recognition model), and each short audio fragment needed to be labeled as either directly generated by AI.

[0057] The AI-generated music recognition model comprises two parts: a feature extractor and a classifier. Given that current AI music generation models typically generate audio directly from a neural network decoder, using traditional spectral features (such as Mel spectrum, Mel frequency cepstral coefficients, linear frequency cepstral coefficients, etc.) in the feature extractor would result in information loss. A more ideal choice is a neural network feature extractor. In this example, RawNet is used as the feature extractor, taking 64,000 sample values ​​as input and outputting multi-channel features encoded by a multi-layer convolutional network, with dimensions of 64×23×43. The classifier uses a neural network classifier; residual networks and gated recurrent units can be used, but a graph neural network classifier is even more ideal. This embodiment uses the AASIST graph neural network as the classifier, receiving a 64×23×43 data layer as input and outputting a value between 0 and 1, representing the probability that the audio segment belongs to AI-generated music. If the probability is greater than a second probability threshold, the singing segment is determined to be AI-generated.

[0058] For example, such as Figure 4 As shown, the song audio and AI music tags are first obtained, and the audio is sliced ​​to obtain multiple audio segments. The audio segments are then input into an AI generation and recognition model consisting of a feature extractor and a classifier. Finally, the recognition result of the singing segment is obtained, that is, the probability that the singing segment belongs to AI generation. If the probability is greater than the first probability threshold, the singing segment is determined to belong to AI generation.

[0059] Optionally, the output probability calculated by the AI-generated recognition model is compared with the input label of the audio segment to calculate the loss. To address potential model training bias caused by imbalanced data, this embodiment uses a binary focus loss function to calculate the loss and employs gradient descent to backpropagate the loss and update the model parameters. Training is stopped when the loss function fails to decrease for several consecutive rounds.

[0060] The above method includes: acquiring multiple third training samples, each carrying a third sample label, which includes a fifth label and a sixth label. The fifth label indicates that the third training sample is an AI cover song audio, and the sixth label indicates that the third training sample is a non-AI cover song audio. Non-AI cover song audio includes normal audio and AI-generated audio. The third training samples are input into the AI ​​cover song recognition model to be trained, and the AI ​​cover song recognition model determines the sample recognition result of the third training samples. Based on the sample classification result and the third sample label of the third training samples, a third loss function value is calculated. Based on the third loss function value, the model parameters of the AI ​​cover song recognition model are adjusted until the third loss function value meets the third preset condition, thereby obtaining the trained AI cover song recognition model.

[0061] The third training sample mentioned above belongs to the first audio segment, that is, the third training sample mentioned above is a singing segment.

[0062] The third training sample was determined as follows: a dataset containing AI-generated cover audio, normal audio, and AI-generated audio was collected. AI-generated cover audio data is the core data and should cover various cover versions where the vocals are replaced using AI technology. Normal audio and AI-generated audio data serve as negative samples to assist the AI ​​cover recognition model in identifying the unique features of AI-generated cover audio. All data was first processed using a pre-trained vocal recognition model to filter out segments containing vocals (i.e., the second training sample). These vocal segments needed to be accurately labeled as AI covers (i.e., the fifth label above) or non-AI covers (i.e., the sixth label above). Non-AI covers were labeled as either "normal audio" or "AI-generated audio."

[0063] To address the characteristics of AI-generated cover audio, this study focuses on detecting any synthetic traces in the vocals while eliminating interference from non-vocal components. This embodiment employs a model structure that connects an audio track segmentation network and a classifier network. First, 64,000 sample values ​​are input at once. An audio feature extraction model converts these values ​​into a 1*257*401 dimension single-dimensional spectrum. Then, the Unet network, which excels in audio track segmentation, is used as the front end to convert the single spectrum into multi-spectral information, outputting 4*257*401. This is subsequently connected to the AASIST graph neural network classifier for identification, outputting a binary classification result to indicate whether the vocal segment is an AI cover.

[0064] For example, such as Figure 5 As shown, the song audio and AI cover song tags are first obtained. The song recognition model outputs a song segment, then features are extracted and input into the AI ​​cover song recognition model, which consists of audio track division and classifier. Finally, the recognition result of the song segment is obtained, that is, the probability that the song segment belongs to the AI ​​cover song. If the probability is greater than the second probability threshold, the song segment is determined to belong to the AI ​​cover song.

[0065] Optionally, the same loss function as the AI ​​song recognition model can be used. Data augmentation is particularly crucial for AI cover song audio data, and can enhance data diversity by making subtle adjustments to the pitch, tempo, reverberation, etc. Furthermore, a loss function or regularization term can be added to penalize instances where the model misclassifies an AI song's vocals as an AI cover.

[0066] Compared with existing technologies, this technical solution has the following advantages: 1. Significantly improves the recognition rate of AI cover songs: This disclosure utilizes a specially trained AI cover song recognition model, combined with a vocal recognition model, to accurately extract vocal fragments, enabling the model to focus on the unique features of AI cover song audio. This approach avoids interference caused by the authenticity of accompaniment in existing technologies, thus more accurately capturing traces of AI-synthesized vocals and their combination with non-AI accompaniment, significantly improving the recognition accuracy of AI cover song audio.

[0067] 2. Effectively reduces the probability of misclassifying AI-generated songs as AI covers: This invention introduces an AI-generated recognition model to judge non-vocal segments. During the comprehensive decision-making stage, if the probability of non-vocal segments of a song being judged as AI-generated audio is high, the song is more likely to be identified as AI-generated audio rather than an AI cover. This mechanism effectively avoids misclassifying songs entirely generated by AI (including AI-generated accompaniment and vocals) as AI covers, thereby reducing the misclassification rate and improving classification accuracy.

[0068] 3. Capable of labeling songs with high confidence (AI-generated, AI-covered, and normal audio): By combining three models and using a divide-and-conquer approach, accurate song labeling can be achieved, reducing the workload of manual review.

[0069] Corresponding to the above method embodiments, this disclosure provides an audio recognition device, such as... Figure 6 As shown, the device includes: The first recognition module 601 is used to acquire the audio to be recognized; and to determine at least one first audio segment and at least one second audio segment from the audio to be recognized using a pre-trained first recognition model; wherein the proportion of singing duration in the first audio segment satisfies a first specified condition, and the proportion of singing duration in the second audio segment satisfies a second specified condition. The second recognition module 602 is used to input the first audio segment and the second audio segment into the pre-trained second recognition model, and determine the first probability of the first audio segment and the second probability of the second audio segment through the second recognition model; the first probability is the probability that the first audio segment belongs to the AI ​​cover audio, and the second probability is the probability that the second audio segment belongs to the AI ​​generated audio. The result determination module 603 is used to determine the audio recognition result of the audio to be recognized based on the first probability and the second probability; wherein the audio recognition result is: AI cover audio, AI generated audio or normal audio.

[0070] This disclosure provides an audio recognition device that acquires audio to be recognized; determines at least one first audio segment and at least one second audio segment from the audio to be recognized using a pre-trained first recognition model; wherein the proportion of singing duration in the first audio segment satisfies a first specified condition, and the proportion of singing duration in the second audio segment satisfies a second specified condition; inputs the first audio segment and the second audio segment into the pre-trained second recognition model, and determines a first probability of the first audio segment and a second probability of the second audio segment using the second recognition model; the first probability is the probability that the first audio segment belongs to AI cover audio, and the second probability is the probability that the second audio segment belongs to AI generated audio; and determines the audio recognition result of the audio to be recognized based on the first probability and the second probability; wherein the audio recognition result is: AI cover audio, AI generated audio, or normal audio. In this method, by determining the singing segment and the non-singing segment of the audio to be identified, the probability of identifying the singing segment as an AI cover audio and the probability of identifying the non-singing segment as an AI-generated audio are determined, and the recognition result of the audio to be identified is determined. By identifying the singing segment and the non-singing segment separately, this method avoids the non-singing segment from misleading the recognition result, improves the recognition accuracy of the singing segment, and thus improves the overall audio recognition accuracy.

[0071] The first recognition module is further configured to: perform audio slicing processing on the audio to be recognized to obtain multiple audio segments; input the multiple audio segments into the first recognition model, and determine at least one first audio segment and at least one second audio segment from the multiple audio segments through the first recognition model.

[0072] The aforementioned second recognition model includes: an AI cover song recognition model and an AI generation recognition model. The aforementioned second recognition module is also used to: input the first audio segment into the AI ​​cover song recognition model and determine the first probability of the first audio segment through the AI ​​cover song recognition model; input the second audio segment into the AI ​​generation recognition model and determine the second probability of the second audio segment through the AI ​​generation recognition model.

[0073] The aforementioned recognition result determination module is further configured to: calculate a first average value of a first probability of at least one first audio segment, calculate a second average value of a second probability of at least one second audio segment, and determine the audio recognition result of the audio to be recognized based on the first average value, the second average value, and preset conditions.

[0074] The aforementioned preset conditions include a first preset condition, a second preset condition, and a third preset condition; the aforementioned recognition result determination module is further configured to: if the first average value and the second average value satisfy the first preset condition, determine that the audio recognition result of the audio to be recognized is an AI cover audio; if the first average value and the second average value satisfy the second preset condition, determine that the audio recognition result of the audio to be recognized is an AI generated audio; if the first average value and the second average value satisfy the third preset condition, determine that the audio recognition result of the audio to be recognized is normal audio.

[0075] The above-mentioned device further includes: a second determining module, used to determine the recognition result of the audio to be recognized as the target recognition result if the first average value and the second average value do not meet the preset conditions, wherein the target recognition result indicates that the audio to be recognized needs to be further recognized based on auxiliary features or needs to be manually recognized.

[0076] The aforementioned apparatus further includes: a first training module, configured to acquire multiple first training samples, each first training sample carrying a first sample label, the first sample label including a first tag and a second tag, the first tag indicating that the first training sample is a singing segment, and the second tag indicating that the first training sample is a non-singing segment; inputting the first training samples into a first recognition model to be trained, and determining the sample classification result of the first training samples through the first recognition model; calculating a first loss function value based on the sample classification result of the first training samples and the first sample label of the first training samples; and adjusting the model parameters of the first recognition model based on the first loss function value until the first loss function value meets a first preset condition, thereby obtaining a trained first recognition model.

[0077] The aforementioned device further includes: a second training module, used to acquire multiple second training samples, each carrying a second sample label, the second sample label including a third label and a fourth label, the third label indicating that the second training sample is AI-generated audio, and the fourth label indicating that the second training sample is normal audio; inputting the second training samples into the AI ​​generation and recognition model to be trained, and determining the sample recognition result of the second training samples through the AI ​​generation and recognition model; calculating a second loss function value based on the sample classification result of the second training samples and the second sample label of the second training samples; and adjusting the model parameters of the AI ​​generation and recognition model based on the second loss function value until the second loss function value meets a second preset condition, thereby obtaining a trained AI generation and recognition model.

[0078] The aforementioned device further includes: a third training module, used to acquire multiple third training samples, each carrying a third sample label, the third sample label including a fifth label and a sixth label, the fifth label indicating that the third training sample is an AI cover song audio, and the sixth label indicating that the third training sample is a non-AI cover song audio, the non-AI cover song audio including normal audio and AI-generated audio; inputting the third training samples into the AI ​​cover song recognition model to be trained, and determining the sample recognition result of the third training samples through the AI ​​cover song recognition model; calculating a third loss function value based on the sample classification result of the third training samples and the third sample label of the third training samples; and adjusting the model parameters of the AI ​​cover song recognition model based on the third loss function value until the third loss function value meets the third preset condition, thereby obtaining the trained AI cover song recognition model.

[0079] The audio recognition device provided in this embodiment has the same technical features as the audio recognition method provided in the above embodiments, so it can also solve the same technical problems and achieve the same technical effects.

[0080] This embodiment also provides an electronic device, including a processor and a memory. The memory stores machine-executable instructions that can be executed by the processor, and the processor executes the machine-executable instructions to implement the above-described audio recognition method. This electronic device can be a server or a terminal device.

[0081] See Figure 7 As shown, the electronic device includes a processor 100 and a memory 101. The memory 101 stores machine-executable instructions that can be executed by the processor 100. The processor 100 executes the machine-executable instructions to implement the above-described audio recognition method.

[0082] Furthermore, Figure 7 The electronic device shown also includes a bus 102 and a communication interface 103, with the processor 100, the communication interface 103 and the memory 101 connected via the bus 102.

[0083] The memory 101 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 103 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc. The bus 102 may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0084] The processor 100 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 100 or by instructions in software form. The processor 100 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this disclosure can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a readily available storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 101, and the processor 100 reads the information from memory 101 and, in conjunction with its hardware, completes the steps of the method described in the foregoing embodiments.

[0085] The processor in the aforementioned electronic device, by executing machine-executable instructions, can perform the following operations in the aforementioned audio recognition method: The process involves: acquiring the audio to be identified; determining at least one first audio segment and at least one second audio segment from the audio using a pre-trained first recognition model; wherein the proportion of singing duration in the first audio segment satisfies a first specified condition, and the proportion of singing duration in the second audio segment satisfies a second specified condition; inputting the first and second audio segments into the pre-trained second recognition model, and determining a first probability of the first audio segment and a second probability of the second audio segment using the second recognition model; the first probability being the probability that the first audio segment belongs to an AI cover song, and the second probability being the probability that the second audio segment belongs to an AI-generated audio; and determining the audio recognition result of the audio to be identified based on the first and second probabilities; wherein the audio recognition result is: AI cover song, AI-generated audio, or normal audio. In this method, by determining the singing segment and the non-singing segment of the audio to be identified, the probability of identifying the singing segment as an AI cover audio and the probability of identifying the non-singing segment as an AI-generated audio are determined, and the recognition result of the audio to be identified is determined. By identifying the singing segment and the non-singing segment separately, this method avoids the non-singing segment from misleading the recognition result, improves the recognition accuracy of the singing segment, and thus improves the overall audio recognition accuracy.

[0086] The steps described above, which involve using a pre-trained first recognition model to determine at least one first audio segment and at least one second audio segment from the audio to be recognized, include: performing audio slicing on the audio to be recognized to obtain multiple audio segments; inputting the multiple audio segments into the first recognition model; and using the first recognition model to determine at least one first audio segment and at least one second audio segment from the multiple audio segments.

[0087] The aforementioned second recognition model includes: an AI cover song recognition model and an AI generation recognition model; the step of inputting a first audio segment and a second audio segment into the pre-trained second recognition model, and determining a first probability of the first audio segment and a second probability of the second audio segment through the second recognition model, includes: inputting the first audio segment into the AI ​​cover song recognition model, and determining a first probability of the first audio segment through the AI ​​cover song recognition model; inputting the second audio segment into the AI ​​generation recognition model, and determining a second probability of the second audio segment through the AI ​​generation recognition model.

[0088] The steps described above for determining the audio recognition result of the audio to be recognized based on the first probability and the second probability include: calculating a first average value of the first probability of at least one first audio segment, calculating a second average value of the second probability of at least one second audio segment; and determining the audio recognition result of the audio to be recognized based on the first average value, the second average value, and preset conditions.

[0089] The aforementioned preset conditions include a first preset condition, a second preset condition, and a third preset condition. The step of determining the audio recognition result of the audio to be recognized based on the first average value, the second average value, and the preset conditions includes: if the first average value and the second average value satisfy the first preset condition, determining the audio recognition result of the audio to be recognized as an AI cover song; if the first average value and the second average value satisfy the second preset condition, determining the audio recognition result of the audio to be recognized as an AI-generated audio; and if the first average value and the second average value satisfy the third preset condition, determining the audio recognition result of the audio to be recognized as normal audio.

[0090] The above method also includes: if the first average value and the second average value do not meet the preset conditions, the recognition result of the audio to be recognized is determined as the target recognition result, and the target recognition result indicates that the audio to be recognized needs to be further recognized based on auxiliary features or needs to be manually recognized.

[0091] The above method includes: acquiring multiple first training samples, each first training sample carrying a first sample label, the first sample label including a first tag and a second tag, the first tag indicating that the first training sample is a singing segment, and the second tag indicating that the first training sample is a non-singing segment; inputting the first training samples into a first recognition model to be trained, and determining the sample classification result of the first training samples through the first recognition model; calculating a first loss function value based on the sample classification result of the first training samples and the first sample label of the first training samples; and adjusting the model parameters of the first recognition model based on the first loss function value until the first loss function value meets a first preset condition, thereby obtaining a trained first recognition model.

[0092] The above method includes: acquiring multiple second training samples, each carrying a second sample label, which includes a third label and a fourth label. The third label indicates that the second training sample is AI-generated audio, and the fourth label indicates that the second training sample is normal audio. The second training samples are input into an AI generation and recognition model to be trained, and the AI ​​generation and recognition model determines the sample recognition result of the second training samples. Based on the sample classification result of the second training samples and the second sample label of the second training samples, a second loss function value is calculated. Based on the second loss function value, the model parameters of the AI ​​generation and recognition model are adjusted until the second loss function value meets a second preset condition, thereby obtaining a trained AI generation and recognition model.

[0093] The above method includes: acquiring multiple third training samples, each carrying a third sample label, which includes a fifth label and a sixth label. The fifth label indicates that the third training sample is an AI cover song audio, and the sixth label indicates that the third training sample is a non-AI cover song audio. Non-AI cover song audio includes normal audio and AI-generated audio. The third training samples are input into the AI ​​cover song recognition model to be trained, and the AI ​​cover song recognition model determines the sample recognition result of the third training samples. Based on the sample classification result and the third sample label of the third training samples, a third loss function value is calculated. Based on the third loss function value, the model parameters of the AI ​​cover song recognition model are adjusted until the third loss function value meets the third preset condition, thereby obtaining the trained AI cover song recognition model.

[0094] This embodiment also provides a machine-readable storage medium storing machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions cause the processor to implement the above-described audio recognition method.

[0095] The machine-executable instructions stored in the aforementioned machine-readable storage medium can be executed to perform the following operations in the aforementioned audio recognition method: The process involves: acquiring the audio to be identified; determining at least one first audio segment and at least one second audio segment from the audio using a pre-trained first recognition model; wherein the proportion of singing duration in the first audio segment satisfies a first specified condition, and the proportion of singing duration in the second audio segment satisfies a second specified condition; inputting the first and second audio segments into the pre-trained second recognition model, and determining a first probability of the first audio segment and a second probability of the second audio segment using the second recognition model; the first probability being the probability that the first audio segment belongs to an AI cover song, and the second probability being the probability that the second audio segment belongs to an AI-generated audio; and determining the audio recognition result of the audio to be identified based on the first and second probabilities; wherein the audio recognition result is: AI cover song, AI-generated audio, or normal audio. In this method, by determining the singing segment and the non-singing segment of the audio to be identified, the probability of identifying the singing segment as an AI cover audio and the probability of identifying the non-singing segment as an AI-generated audio are determined, and the recognition result of the audio to be identified is determined. By identifying the singing segment and the non-singing segment separately, this method avoids the non-singing segment from misleading the recognition result, improves the recognition accuracy of the singing segment, and thus improves the overall audio recognition accuracy.

[0096] The steps described above, which involve using a pre-trained first recognition model to determine at least one first audio segment and at least one second audio segment from the audio to be recognized, include: performing audio slicing on the audio to be recognized to obtain multiple audio segments; inputting the multiple audio segments into the first recognition model; and using the first recognition model to determine at least one first audio segment and at least one second audio segment from the multiple audio segments.

[0097] The aforementioned second recognition model includes: an AI cover song recognition model and an AI generation recognition model; the step of inputting a first audio segment and a second audio segment into the pre-trained second recognition model, and determining a first probability of the first audio segment and a second probability of the second audio segment through the second recognition model, includes: inputting the first audio segment into the AI ​​cover song recognition model, and determining a first probability of the first audio segment through the AI ​​cover song recognition model; inputting the second audio segment into the AI ​​generation recognition model, and determining a second probability of the second audio segment through the AI ​​generation recognition model.

[0098] The steps described above for determining the audio recognition result of the audio to be recognized based on the first probability and the second probability include: calculating a first average value of the first probability of at least one first audio segment, calculating a second average value of the second probability of at least one second audio segment; and determining the audio recognition result of the audio to be recognized based on the first average value, the second average value, and preset conditions.

[0099] The aforementioned preset conditions include a first preset condition, a second preset condition, and a third preset condition. The step of determining the audio recognition result of the audio to be recognized based on the first average value, the second average value, and the preset conditions includes: if the first average value and the second average value satisfy the first preset condition, determining the audio recognition result of the audio to be recognized as an AI cover song; if the first average value and the second average value satisfy the second preset condition, determining the audio recognition result of the audio to be recognized as an AI-generated audio; and if the first average value and the second average value satisfy the third preset condition, determining the audio recognition result of the audio to be recognized as normal audio.

[0100] The above method also includes: if the first average value and the second average value do not meet the preset conditions, the recognition result of the audio to be recognized is determined as the target recognition result, and the target recognition result indicates that the audio to be recognized needs to be further recognized based on auxiliary features or needs to be manually recognized.

[0101] The above method includes: acquiring multiple first training samples, each first training sample carrying a first sample label, the first sample label including a first tag and a second tag, the first tag indicating that the first training sample is a singing segment, and the second tag indicating that the first training sample is a non-singing segment; inputting the first training samples into a first recognition model to be trained, and determining the sample classification result of the first training samples through the first recognition model; calculating a first loss function value based on the sample classification result of the first training samples and the first sample label of the first training samples; and adjusting the model parameters of the first recognition model based on the first loss function value until the first loss function value meets a first preset condition, thereby obtaining a trained first recognition model.

[0102] The above method includes: acquiring multiple second training samples, each carrying a second sample label, which includes a third label and a fourth label. The third label indicates that the second training sample is AI-generated audio, and the fourth label indicates that the second training sample is normal audio. The second training samples are input into an AI generation and recognition model to be trained, and the AI ​​generation and recognition model determines the sample recognition result of the second training samples. Based on the sample classification result of the second training samples and the second sample label of the second training samples, a second loss function value is calculated. Based on the second loss function value, the model parameters of the AI ​​generation and recognition model are adjusted until the second loss function value meets a second preset condition, thereby obtaining a trained AI generation and recognition model.

[0103] The above method includes: acquiring multiple third training samples, each carrying a third sample label, which includes a fifth label and a sixth label. The fifth label indicates that the third training sample is an AI cover song audio, and the sixth label indicates that the third training sample is a non-AI cover song audio. Non-AI cover song audio includes normal audio and AI-generated audio. The third training samples are input into the AI ​​cover song recognition model to be trained, and the AI ​​cover song recognition model determines the sample recognition result of the third training samples. Based on the sample classification result and the third sample label of the third training samples, a third loss function value is calculated. Based on the third loss function value, the model parameters of the AI ​​cover song recognition model are adjusted until the third loss function value meets the third preset condition, thereby obtaining the trained AI cover song recognition model.

[0104] The computer program products of the audio recognition method, apparatus, electronic device and system provided in the embodiments of this disclosure include a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the preceding method embodiments. For specific implementation, please refer to the method embodiments, which will not be repeated here.

[0105] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system and apparatus described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0106] Furthermore, in the description of the embodiments of this disclosure, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this disclosure based on the specific circumstances.

[0107] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0108] In the description of this disclosure, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this disclosure and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this disclosure. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0109] Finally, it should be noted that the above embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. An audio recognition method, characterized in that, The method includes: Acquire the audio to be identified; determine at least one first audio segment and at least one second audio segment from the audio to be identified using a pre-trained first recognition model; wherein the proportion of singing duration in the first audio segment satisfies a first specified condition, and the proportion of singing duration in the second audio segment satisfies a second specified condition; The first audio segment and the second audio segment are input into a pre-trained second recognition model. The second recognition model determines a first probability of the first audio segment and a second probability of the second audio segment. The first probability is the probability that the first audio segment belongs to AI cover audio, and the second probability is the probability that the second audio segment belongs to AI generated audio. Based on the first probability and the second probability, the audio recognition result of the audio to be recognized is determined; wherein, the audio recognition result is: AI cover audio, AI generated audio, or normal audio.

2. The method according to claim 1, characterized in that, The step of determining at least one first audio segment and at least one second audio segment from the audio to be identified using a pre-trained first recognition model includes: The audio to be identified is processed into audio segments to obtain multiple audio fragments; The plurality of audio segments are input into the first recognition model, and the first recognition model is used to determine the at least one first audio segment and the at least one second audio segment from the plurality of audio segments.

3. The method according to claim 1, characterized in that, The second recognition model includes: an AI cover song recognition model and an AI-generated recognition model; The step of inputting the first audio segment and the second audio segment into a pre-trained second recognition model, and determining the first probability of the first audio segment and the second probability of the second audio segment through the second recognition model, includes: The first audio segment is input into the AI ​​cover song recognition model, and the first probability of the first audio segment is determined by the AI ​​cover song recognition model. The second audio segment is input into the AI-generated recognition model, and the second probability of the second audio segment is determined by the AI-generated recognition model.

4. The method according to claim 1, characterized in that, The step of determining the audio recognition result of the audio to be recognized based on the first probability and the second probability includes: Calculate a first average value of the first probability of the at least one first audio segment, and calculate a second average value of the second probability of the at least one second audio segment; The audio recognition result of the audio to be recognized is determined based on the first average value, the second average value, and the preset conditions.

5. The method according to claim 4, characterized in that, The preset conditions include a first preset condition, a second preset condition, and a third preset condition; The step of determining the audio recognition result of the audio to be recognized based on the first average value, the second average value, and preset conditions includes: If the first average value and the second average value satisfy the first preset condition, the audio recognition result of the audio to be identified is determined to be the AI ​​cover audio. If the first average value and the second average value satisfy the second preset condition, the audio recognition result of the audio to be recognized is determined to be the AI-generated audio; If the first average value and the second average value satisfy the third preset condition, the audio recognition result of the audio to be recognized is determined to be normal audio.

6. The method according to claim 4, characterized in that, The method further includes: If the first average value and the second average value do not meet the preset conditions, the recognition result of the audio to be recognized is determined to be the target recognition result. The target recognition result indicates that the audio to be recognized needs to be further recognized based on auxiliary features or needs to be manually recognized.

7. The method according to claim 1, characterized in that, The method includes: Multiple first training samples are obtained. Each first training sample carries a first sample label. The first sample label includes a first label and a second label. The first label is used to indicate that the first training sample is a singing segment, and the second label is used to indicate that the first training sample is a non-singing segment. The first training sample is input into the first recognition model to be trained, and the first recognition model determines the sample classification result of the first training sample. Based on the sample classification result of the first training sample and the first sample label of the first training sample, calculate the first loss function value; Based on the first loss function value, the model parameters of the first recognition model are adjusted until the first loss function value meets the first preset condition, thus obtaining the trained first recognition model.

8. An audio recognition device, characterized in that, The device includes: The first recognition module is used to acquire the audio to be recognized; and to determine at least one first audio segment and at least one second audio segment from the audio to be recognized using a pre-trained first recognition model; wherein the proportion of singing duration in the first audio segment satisfies a first specified condition, and the proportion of singing duration in the second audio segment satisfies a second specified condition. The second recognition module is used to input the first audio segment and the second audio segment into a pre-trained second recognition model, and determine a first probability of the first audio segment and a second probability of the second audio segment through the second recognition model; the first probability is the probability that the first audio segment belongs to AI cover audio, and the second probability is the probability that the second audio segment belongs to AI generated audio; The result determination module is used to determine the audio recognition result of the audio to be recognized based on the first probability and the second probability; wherein the audio recognition result is: AI cover audio, AI generated audio, or normal audio.

9. An electronic device, characterized in that, The device includes a processor and a memory, the memory storing computer-executable instructions that can be executed by the processor, the processor executing the computer-executable instructions to implement the audio recognition method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when invoked and executed by a processor, cause the processor to implement the audio recognition method according to any one of claims 1-7.

Images