Audio recognition model training method and device, storage medium and electronic equipment

Abstract

The invention relates to the technical field of artificial intelligence, and provides an audio recognition model training method, an audio recognition model training device, a computer readable storage medium and electronic equipment, and the method comprises the steps: obtaining a plurality of groups of training samples, and collecting each piece of frequency spectrum information corresponding to each audio in each group of training samples; extracting each spectrum feature through a feature extraction network, and encoding each spectrum feature through an audio encoder to obtain each audio vector; determining a comparison loss value corresponding to each group of training samples according to the difference degree between the audio vectors corresponding to each group of training samples, and performing iterative training on the feature extraction network and the audio encoder according to multiple groups of comparison loss values; and when the comparison loss value satisfies a first preset convergence condition, freezing the feature extraction network and the audio encoder, and performing iterative training on the classifier according to the audio vectors corresponding to the multiple groups of training samples to obtain an audio recognition model. The audio recognition accuracy can be improved.

Description

Background Art

[0002] As a major branch of artificial intelligence, audio synthesis technology has made significant progress in recent years with the breakthrough of deep learning technology. Early audio synthesis models such as Tacotron and WaveNet have been able to generate highly realistic synthesized speech. Therefore, how to identify forged audio has become a focus of attention for relevant technical personnel.

[0003] In the related art, forged audio is generally identified based on a binary classification model. However, when the difference between the synthesized audio and the real audio becomes smaller and smaller, the above solution cannot accurately identify the forged audio.

[0004] In view of this, there is an urgent need in the art to develop a new training method and device for an audio recognition model.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure. Summary of the invention

[0006] The purpose of the present disclosure is to provide a training method for an audio recognition model, a training device for an audio recognition model, a computer-readable storage medium and an electronic device, thereby overcoming, at least to a certain extent, the technical problem of being unable to accurately identify forged audio due to limitations of related technologies.

[0007] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0008] According to a first aspect of the present disclosure, a method for training an audio recognition model is provided, comprising:

[0009] Acquire multiple groups of training samples, and collect each spectrum information corresponding to each audio contained in each group of training samples; each group of training samples contains positive sample audio and negative sample audio, the positive sample audio contains original audio and n-1 enhanced audio, and the negative sample audio contains n forged audio, where n is an integer greater than 1;

[0010] Extracting each spectral feature corresponding to each spectral information through a feature extraction network of the initial audio recognition model, encoding each spectral feature through an audio encoder of the initial audio recognition model to obtain each audio vector;

[0011] Determining a contrast loss value corresponding to each group of training samples according to the degree of difference between the audio vectors corresponding to each group of training samples, and iteratively training the feature extraction network and the audio encoder according to multiple groups of the contrast loss values;

[0012] When the contrast loss value meets a first preset convergence condition, the feature extraction network and the audio encoder are frozen, and the classifier of the initial audio recognition model is iteratively trained according to the audio vectors corresponding to the multiple groups of training samples to obtain a trained audio recognition model.

[0013] In an exemplary embodiment of the present disclosure, each group of training samples is obtained by:

[0014] Collecting each original audio from an audio resource library;

[0015] Performing audio enhancement processing on each of the original audios to obtain n-1 enhanced audios corresponding to each of the original audios;

[0016] Audio synthesis processing is performed on each original audio through multiple audio synthesis models to obtain n forged audios corresponding to each original audio.

[0017] In an exemplary embodiment of the present disclosure, the step of collecting each spectrum information corresponding to each audio contained in each group of training samples includes:

[0018] Performing frame processing on each audio to obtain multiple audio frames;

[0019] Perform Fourier transform on each audio frame to obtain each frequency domain signal;

[0020] Input the obtained multiple frequency domain signals into the Mel filter to obtain the Mel spectrum;

[0021] Perform logarithmic transformation on the Mel spectrum to obtain each spectrum information corresponding to each audio.

[0022] In an exemplary embodiment of the present disclosure, the audio vector corresponding to each group of training samples includes a positive sample vector corresponding to each positive sample audio and a negative sample vector corresponding to each negative sample audio;

[0023] The determining, according to the degree of difference between the audio vectors corresponding to each group of training samples, the contrast loss value corresponding to each group of training samples comprises:

[0024] Determine the contrast loss value corresponding to each group of training samples according to the inner product between each positive sample vector and the transposed vector of the target positive sample vector, the inner product between each positive sample vector and the transposed vector of each negative sample vector, the inner product between each negative sample vector and the transposed vector of the target negative sample vector, and the inner product between each negative sample vector and the transposed vector of each positive sample vector;

[0025] The target positive sample vector is randomly selected from n-1 positive sample vectors other than the positive sample vector, and the target negative sample vector is randomly selected from n-1 negative sample vectors other than the negative sample vector.

[0026] In an exemplary embodiment of the present disclosure, determining the contrast loss value corresponding to each group of training samples according to the inner product between each positive sample vector and the transposed vector of the target positive sample vector, the inner product between each positive sample vector and the transposed vector of each negative sample vector, the inner product between each negative sample vector and the transposed vector of the target negative sample vector, and the inner product between each negative sample vector and the transposed vector of each positive sample vector includes:

[0027] The contrast loss value is determined based on the following formula:

[0028]

[0029] Among them, L a represents the contrast loss value, t represents a preset temperature coefficient, and n represents the number of the positive sample audio or the negative sample audio in each group of training samples;

[0030] r i Represents each positive sample vector, r′ z A transposed vector representing the target positive sample vector;

[0031] f i Represents each negative sample vector, f′ z represents the transposed vector of the target negative sample vector.

[0032] In an exemplary embodiment of the present disclosure, iteratively training the classifier of the initial audio recognition model according to the audio vectors corresponding to the multiple groups of training samples to obtain a trained audio recognition model includes:

[0033] Performing audio recognition processing on each audio vector corresponding to each audio in each group of training samples by using the classifier of the initial audio recognition model to obtain a predicted label corresponding to each audio;

[0034] Obtaining a true label corresponding to each audio, determining a classification loss value corresponding to each group of training samples according to a difference value between the true label corresponding to each audio and the predicted label, and updating a network parameter of the classifier according to the classification loss value;

[0035] When the classification loss value satisfies a second preset convergence condition, a trained audio recognition model is obtained.

[0036] In an exemplary embodiment of the present disclosure, determining the classification loss value corresponding to each group of training samples according to the difference value between the real label corresponding to each audio and the predicted label includes:

[0037] The classification loss value corresponding to each group of training samples is calculated based on the following formula:

[0038]

[0039] Among them, L b represents the classification loss value, N = 2n; y i represents the true label corresponding to each audio, Indicates the predicted label corresponding to each audio.

[0040] In an exemplary embodiment of the present disclosure, after obtaining the trained audio recognition model, the method further includes:

[0041] Acquire the audio to be recognized, and collect the frequency spectrum information to be recognized corresponding to the audio to be recognized;

[0042] Inputting the frequency spectrum information to be recognized into the trained audio recognition model, and outputting the audio recognition result through the trained audio recognition model;

[0043] The audio recognition result is used to indicate whether the audio to be recognized is forged audio.

[0044] According to a second aspect of the present disclosure, there is provided a training device for an audio recognition model, comprising:

[0045] A sample acquisition module is used to acquire multiple groups of training samples and collect each spectrum information corresponding to each audio contained in each group of training samples; each group of training samples includes positive sample audio and negative sample audio, the positive sample audio includes original audio and n-1 enhanced audio, and the negative sample audio includes n forged audio;

[0046] A feature extraction and encoding module, used to extract each spectrum feature corresponding to each spectrum information through a feature extraction network of the initial audio recognition model, and encode each spectrum feature through an audio encoder of the initial audio recognition model to obtain each audio vector;

[0047] A first training module, configured to determine a contrast loss value corresponding to each group of training samples according to a degree of difference between audio vectors corresponding to each group of training samples, and iteratively train the feature extraction network and the audio encoder according to the contrast loss value;

[0048] The second training module is used to freeze the feature extraction network and the audio encoder when the contrast loss value meets the first preset convergence condition, and iteratively train the classifier of the initial audio recognition model according to the audio vector corresponding to each group of training samples to obtain a trained audio recognition model.

[0049] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the training method of the audio recognition model described in the first aspect is implemented.

[0050] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the training method of the audio recognition model described in the first aspect above by executing the executable instructions.

[0051] It can be seen from the above technical solutions that the audio recognition model training method, audio recognition model training device, computer-readable storage medium and electronic device in the exemplary embodiments of the present disclosure have at least the following advantages and positive effects:

[0052] In the technical solutions provided in some embodiments of the present disclosure, on the one hand, by including positive sample audio and negative sample audio in the training samples, the positive sample audio includes the original audio and n-1 enhanced audios, which increases the diversity of the training samples, so that the model can learn the audio features under different conditions and enhance the adaptability of the model to real-world audio changes. The negative sample audio includes n forged audios. By introducing forged audios as negative samples, the model can distinguish the difference between real audio and forged audio, thereby improving the anti-deception ability of the model in practical applications. Furthermore, through the joint work of the feature extraction network and the audio encoder, representative spectral features are extracted from the spectral information, and these features are encoded into compact audio vectors. By calculating and minimizing the contrastive loss value, the model can gradually optimize its internal representation to ensure that different categories of audio are clearly distinguishable in the feature space; further, by minimizing the contrastive loss value, the model can effectively narrow the distance between similar categories in the feature space, while pushing the distance between different categories apart, so that the model can learn the difference between real audio and fake audio during training; on the other hand, by freezing these two parts when the contrastive loss value meets the preset convergence conditions and focusing on the training of the classifier, this strategy ensures that the model's basic feature extraction and encoding capabilities are strong enough, thereby avoiding the problem of overfitting, while also providing high-quality input for the classifier and improving the accuracy of the final classification.

[0053] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0055] Figure 1 A schematic diagram showing a flow chart of a method for training an audio recognition model in an embodiment of the present disclosure;

[0056] Figure 2 A flowchart showing how to obtain each group of training samples in an embodiment of the present disclosure;

[0057] Figure 3 A schematic diagram showing a flow chart of how to construct training samples in an embodiment of the present disclosure;

[0058] Figure 4 A schematic diagram showing a flow chart of how to collect each spectrum information corresponding to each audio contained in each group of training samples in an embodiment of the present disclosure;

[0059] Figure 5 A schematic diagram showing a flow chart of a comparative learning training phase in an embodiment of the present disclosure is shown;

[0060] Figure 6 A flowchart showing how to iteratively train a classifier of an initial audio recognition model according to audio vectors corresponding to multiple groups of training samples to obtain a trained audio recognition model in an embodiment of the present disclosure;

[0061] Figure 7 A schematic diagram showing the process flow of the model fine-tuning stage in an embodiment of the present disclosure;

[0062] Figure 8 A schematic diagram showing a flow chart of the model application stage in an embodiment of the present disclosure;

[0063] Fig. 9 An overall schematic diagram of the audio recognition model training method in an embodiment of the present disclosure is shown;

[0064] Fig.10 A schematic diagram showing the structure of a training device for an audio recognition model in an exemplary embodiment of the present disclosure;

[0065] Fig.11 A schematic structural diagram of an electronic device in an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0066] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0067] The terms "a", "an", "the" and "said" are used in this specification to indicate the presence of one or more elements / components / etc.; the terms "including" and "having" are used to express an open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first" and "second" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0068] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings represent the same or similar parts, and their repeated descriptions will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0069] As a major branch of artificial intelligence, audio synthesis technology has made significant progress in recent years with the breakthrough of deep learning technology. Early audio synthesis models such as Tacotron and WaveNet have been able to generate highly realistic synthesized speech. A significant application of this type of technology is voice cloning technology, which generates synthesized speech that can imitate the voice of a target individual by analyzing the sound data of a small amount of the target individual. With the further development of the model, today's speech synthesis model can generate highly realistic synthesized audio in a very short time with a very small amount of training data.

[0070] Voice cloning technology has good application prospects in some specific fields, such as helping patients with speech disorders to speak, or providing customized voices for personalized voice assistants. However, technological advances also bring potential threats. As the realism of synthetic voices becomes higher and higher, traditional audio recognition methods have become difficult to distinguish between real and fake audio.

[0071] Currently, the research on audio forgery identification is still in its early stages, and many existing solutions rely mainly on binary classification models (i.e., determining whether the audio is authentic) and are mainly based on surface audio features for analysis. The limitation of this approach is that when audio synthesis technology becomes more complex and the difference between synthesized audio and real audio becomes smaller and smaller, traditional classification models cannot accurately identify forged audio. In addition, existing technologies usually do not have a recognition mechanism specifically for voice cloned audio, and the challenge of voice forgery identification is one of the difficulties in the field of speech synthesis today.

[0072] In order to address the above problems, the present disclosure provides a timbre synthesis forgery recognition model based on contrastive learning, which combines multiple data enhancement and forged audio generation strategies to enhance the model's ability to recognize audio authenticity through contrastive learning methods. The contrastive learning strategy introduced in the present disclosure allows the model to more comprehensively recognize the difference between synthetic audio and real audio, and through data enhancement, the generalization ability of the model can be improved.

[0073] The solution provided by this disclosure at least achieves the following technical objectives:

[0074] First, generate a dataset for contrastive learning: In order to make the model more generalizable, we generate fake audio using a variety of different audio synthesis techniques to build a high-quality contrastive learning dataset. In addition, by enhancing the real audio (such as adding noise, time scaling, etc.), the diversity of the dataset is further improved, making the model more robust in dealing with different types of audio forgeries.

[0075] Second, improve the ability to identify voice forgeries: Most current audio forgery identification models rely on specific binary classification methods. Although this method can distinguish real audio from forged audio to a certain extent, it often loses its effectiveness when faced with more complex voice cloning technology. Through contrastive learning technology, the model can learn the features of real audio and synthetic audio at the same time, and can more accurately learn the subtle feature differences in the audio, thereby establishing a more reliable distinction mechanism between real audio and forged audio;

[0076] Third, reduce the incidence of fraud based on speech synthesis: By building an efficient voice forgery recognition system, the success rate of forged voice being used for fraud can be significantly reduced, especially in the fields of finance, communications, etc., thereby improving the overall security and trust of society.

[0077] In the embodiments of the present disclosure, firstly, a method for training an audio recognition model is provided, which at least to some extent overcomes the defect in the related art that forged audio cannot be accurately identified.

[0078] Figure 1A flow chart of a method for training an audio recognition model in an embodiment of the present disclosure is shown. The execution subject of the method for training an audio recognition model may be a server that trains the audio recognition model.

[0079] refer to Figure 1 According to an embodiment of the present disclosure, a method for training an audio recognition model includes the following steps:

[0080] Step S110, obtaining multiple groups of training samples, and collecting each spectrum information corresponding to each audio contained in each group of training samples; each group of training samples includes positive sample audio and negative sample audio, the positive sample audio includes original audio and n-1 enhanced audios, and the negative sample audio includes n forged audios;

[0081] Step S120, extracting each spectral feature corresponding to each spectral information through the feature extraction network of the initial audio recognition model, encoding each spectral feature through the audio encoder of the initial audio recognition model, and obtaining each audio vector;

[0082] Step S130, determining a contrast loss value corresponding to each group of training samples according to the degree of difference between the audio vectors corresponding to each group of training samples, and iteratively training the feature extraction network and the audio encoder according to the contrast loss value;

[0083] Step S140, when the contrast loss value meets the first preset convergence condition, freeze the feature extraction network and the audio encoder, iteratively train the classifier of the initial audio recognition model according to the audio vector corresponding to each group of training samples, and obtain a trained audio recognition model.

[0084] exist Figure 1In the technical solution provided by the illustrated embodiment, on the one hand, by including positive sample audio and negative sample audio in the training samples, the positive sample audio includes the original audio and n-1 enhanced audios, which increases the diversity of the training samples, so that the model can learn the audio features under different conditions and enhance the model's adaptability to real-world audio changes. The negative sample audio includes n forged audios. By introducing forged audios as negative samples, the model can distinguish the difference between real audio and forged audio, thereby improving the model's anti-deception ability in practical applications. Furthermore, through the joint work of the feature extraction network and the audio encoder, representative spectral features are extracted from the spectral information, and these features are encoded into compact audio vectors. By calculating and minimizing the contrastive loss value, the model can gradually optimize its internal representation to ensure that different categories of audio are clearly distinguishable in the feature space; further, by minimizing the contrastive loss value, the model can effectively narrow the distance between similar categories in the feature space, while pushing the distance between different categories apart, so that the model can learn the difference between real audio and fake audio during training; on the other hand, by freezing these two parts when the contrastive loss value meets the preset convergence conditions and focusing on the training of the classifier, this strategy ensures that the model's basic feature extraction and encoding capabilities are strong enough, thereby avoiding the problem of overfitting, while also providing high-quality input for the classifier and improving the accuracy of the final classification.

[0085] The following Figure 1 The specific implementation process of each step is described in detail:

[0086] In step S110, multiple groups of training samples are obtained, and each spectrum information corresponding to each audio contained in each group of training samples is collected.

[0087] In this step, multiple groups of training samples may be obtained, and each spectrum information corresponding to each audio contained in each group of training samples may be collected.

[0088] Each group of training samples may include positive sample audio and negative sample audio, the positive sample audio includes original audio and n-1 enhanced audios, the negative sample audio includes n forged audios, and n is an integer greater than 1.

[0089] Among them, original audio refers to a real recording without any modification or processing. It is a sound file recorded directly from the sound source, retaining all the natural characteristics and properties of the recording, without synthesis, editing or other forms of human intervention. Enhanced audio refers to an audio version that changes its quality or specific characteristics by applying various audio processing techniques to the original audio (such as randomly adding noise, time scaling, time shifting, etc.). Forged audio refers to audio content that is deliberately created or modified through technical means in order to imitate real audio recordings, but in fact these audios have never been recorded authentically or their content has been tampered with.

[0090] Exemplary, reference Figure 2 , Figure 2 A flowchart showing how to obtain each set of training samples in the embodiment of the present disclosure includes steps S201 to S203:

[0091] In step S201, each original audio is collected from an audio resource library.

[0092] In this step, the audio resource library refers to a legal audio library, such as a commercial audio library, a public domain audio library, an audio library provided by the government or an educational institution, etc. You can choose one based on the actual situation, and this disclosure does not make any special restrictions on this.

[0093] Therefore, the present disclosure can collect each original audio from the above-mentioned legal audio library, and these original audios will be used as benchmarks for model comparison learning to ensure that the audio used is original and unprocessed in terms of voice content, speaker, and environmental sound effects.

[0094] In addition, it should be noted that the original audio needs to carry correct transcription information to facilitate subsequent synthesis processing. Transcription information refers to the process and result of converting the spoken content in the audio into written text. Specifically, transcription information is a text record of the audio content, which accurately reflects every word and sentence spoken in the audio, and sometimes also includes the speaker's tone, pauses, laughter and other non-verbal elements.

[0095] In step S202, audio enhancement processing is performed on each original audio to obtain n-1 enhanced audios corresponding to each original audio.

[0096] In this step, audio enhancement processing can be performed on each original audio to obtain n-1 enhanced audios corresponding to each original audio. The audio enhancement processing can be a combination of one or more of random noise addition, time scaling, timbre replacement, time stretching, time shifting, etc., which can be set according to actual conditions and is not specifically limited in this disclosure.

[0097] These enhanced audios are generated to increase the diversity of the data and enable the model to learn more audio features under environmental noise, thereby improving its robustness in different scenarios.

[0098] In step S203, audio synthesis processing is performed on each original audio through multiple audio synthesis models to obtain n forged audios corresponding to each original audio.

[0099] In this step, forged audio can be generated through a variety of speech synthesis models (such as GPT-SoVits, CosyVoice, etc.). To ensure the quality of the forged audio, the content of the synthesized audio can be consistent with the original audio. These forged audios will be used as negative samples in subsequent comparative learning to help the model learn the characteristics of the forged timbre.

[0100] By setting each group of training samples to include one original audio, n-1 enhanced audios and n forged audios, the present disclosure can ensure that the model can learn effective audio distinguishing features through the comparison of positive and negative samples in each training.

[0101] refer to Figure 3 , Figure 3 A flowchart showing how to construct a training sample in an embodiment of the present disclosure includes steps S301 to S304:

[0102] In step S301, original audio is collected;

[0103] In step S302, data enhancement is performed on the original audio to generate enhanced audio;

[0104] In step S303, the original audio is used to generate synthetic audio through different audio cloning models;

[0105] In step S304, each group of training samples is formed, and each group of training samples includes positive sample audio (original audio and n-1 enhanced audios) and negative sample audio (n forged audios).

[0106] After obtaining each group of training samples, it can be known that each group of training samples in the present disclosure contains 2n audios, of which n are positive sample audios (i.e., one original audio and n-1 enhanced audios) and n are negative sample audios (i.e., n forged audios). Therefore, for each audio, its spectral information can be collected.

[0107] refer to Figure 4 , Figure 4 A flow chart showing how to collect each spectrum information corresponding to each audio contained in each group of training samples in the embodiment of the present disclosure includes steps S401 to S404:

[0108] In step S401, each audio is subjected to frame processing to obtain a plurality of audio frames.

[0109] In this step, each audio can be framed by a sliding window to obtain multiple audio frames. Framing refers to dividing the continuous audio signal into a series of shorter time segments or "frames" to facilitate more detailed analysis and processing of audio data. Each audio frame usually contains hundreds of sample points, representing the information within a small audio time window. Framing is a basic step in many audio processing technologies, especially in the fields of speech recognition, audio coding, feature extraction, etc.

[0110] In step S402, Fourier transform is performed on each audio frame to obtain each frequency domain signal.

[0111] In this step, each audio frame can be subjected to a fast Fourier transform to obtain each frequency domain signal. Among them, the Fast Fourier Transform (FFT) is an efficient algorithm for calculating the Discrete Fourier Transform (DFT) and its inverse transform. FFT is widely used in signal processing, audio analysis and other engineering fields because it can convert signals in the time domain to the frequency domain, making analysis and processing more intuitive and efficient.

[0112] In step S403, the obtained multiple frequency domain signals are input into a Mel filter to obtain a Mel spectrum.

[0113] In this step, the obtained multiple frequency domain signals can be input into a set of Mel filters to obtain Mel spectra. Through the Mel filter, the audio signal can be converted into a two-dimensional representation of time-frequency, so that the model can extract the core features of the audio from the spectrum.

[0114] In step S404, the Mel spectrum is logarithmically transformed to obtain each spectrum information corresponding to each audio.

[0115] In this step, the obtained Mel spectrum may be logarithmically transformed to obtain each spectrum information corresponding to each audio, thereby simulating human perception of audio signal strength.

[0116] After obtaining each spectrum information corresponding to each audio, you can then refer to Figure 1 In step S120, each spectrum feature corresponding to each spectrum information is extracted through the feature extraction network of the initial audio recognition model, and each spectrum feature is encoded through the audio encoder of the initial audio recognition model to obtain each audio vector.

[0117] In this step, each spectral feature corresponding to each spectral information can be extracted through the feature extraction network of the initial audio recognition model, and each spectral feature can be encoded through the audio encoder of the initial audio recognition model to obtain each audio vector.

[0118] Among them, the feature extraction network of the above-mentioned initial audio recognition model can be a convolutional neural network, so that each spectrum information corresponding to each audio can be input into the convolutional neural network, and the spectrum features corresponding to each spectrum information are extracted through the convolutional neural network. The convolutional neural network can capture the features in the Mel spectrum graph and can reduce the number of features so that we can complete the training with fewer resources.

[0119] Afterwards, each obtained spectral feature can be input into the audio encoder of the above-mentioned initial audio recognition model, and the audio encoder can be responsible for converting each spectral feature into an embedded vector (i.e., an audio vector) of fixed dimension. Exemplarily, HTSAT can be used as an audio encoder, the core of which is a multi-layer module based on Swin Transformer, with strong feature extraction capabilities. In this way, the high-dimensional spectral data of the audio is compressed into an embedded representation of a fixed dimension, which is convenient for subsequent comparative learning.

[0120] Thus, after step S120, for the 2n audios contained in each group of training samples, the present invention can obtain 2n audio vectors, of which n are positive sample vectors corresponding to the positive sample audios (i.e., an audio vector of the original audio and n-1 audio vectors of the enhanced audios), and n are negative sample vectors corresponding to the negative sample audios (i.e., n audio vectors of the forged audios).

[0121] After obtaining each audio vector corresponding to each audio, step S130 can be entered to determine the contrast loss value corresponding to each group of training samples according to the degree of difference between the audio vectors corresponding to each group of training samples, and iteratively train the feature extraction network and the audio encoder according to the contrast loss value.

[0122] In this step, the feature extraction network and the audio encoder may be iteratively trained by contrastive learning, wherein the core of contrastive learning is to maximize the inner product between similar samples and minimize the inner product between dissimilar samples.

[0123] Specifically, in the present disclosure, the contrast loss value corresponding to each group of training samples can be determined according to the degree of difference between the audio vectors corresponding to each group of training samples, and the feature extraction network and the audio encoder can be iteratively trained according to multiple groups of contrast loss values.

[0124] In view of the fact that the audio vector corresponding to each group of training samples includes a positive sample vector corresponding to each positive sample audio and a negative sample vector corresponding to each negative sample audio, the present disclosure can determine the contrast loss value corresponding to each group of training samples according to the inner product between each positive sample vector and the transposed vector of the target positive sample vector, the inner product between each positive sample vector and the transposed vector of each negative sample vector, the inner product between each negative sample vector and the transposed vector of the target negative sample vector, and the inner product between each negative sample vector and the transposed vector of each positive sample vector, wherein the target positive sample vector is randomly selected from n-1 positive sample vectors other than the positive sample vector, and the target negative sample vector is randomly selected from n-1 negative sample vectors other than the negative sample vector.

[0125] Specifically, the contrast loss value corresponding to each group of training samples can be determined by the following formula 1:

[0126]

[0127] Among them, L a represents the contrast loss value corresponding to each group of training samples, t is an adjustable temperature coefficient, the specific value of which can be set according to actual conditions, and the present disclosure does not make any special restrictions on this, and n represents the number of positive sample vectors or negative sample vectors in a group of training samples;

[0128] r i Represents each positive sample vector, r′ z Represents the transposed vector of the target positive sample vector, which is the sum of the vectors except r i One of the n-1 positive sample vectors other than ; f′ j Represents the transposed vector of each negative sample vector;

[0129] f i Represents each negative sample vector, f′ z Represents the transposed vector of the target negative sample vector, which is the negative sample vector except f i One of the n-1 negative sample vectors other than ; r′ j Represents the transposed vector of each positive sample vector.

[0130] Specifically, in the actual model training process, a group of training samples can be input each time. After the contrast loss value corresponding to the group is calculated, the network parameters of the above-mentioned feature extraction network and the network parameters of the above-mentioned audio encoder can be updated according to the contrast loss value corresponding to the group of training samples. After that, the next group of training samples can be input, and each spectral feature corresponding to each spectral information in the next group of training samples can be extracted by using the feature extraction network after the parameter update, and each spectral feature can be encoded by using the audio encoder after the parameter update to obtain each audio vector. Then, the next contrast loss value corresponding to the next group of training samples is determined according to the degree of difference between the audio vectors corresponding to the next group of training samples, and the network parameters of the above-mentioned feature extraction network and the network parameters of the above-mentioned audio encoder are updated again according to the next group of contrast loss values, so as to perform multiple iterative training on the feature extraction network and the audio encoder of the above-mentioned initial audio recognition model, until a certain contrast loss value obtained satisfies the first preset convergence condition (for example: less than or equal to the preset loss value), and the iterative training process can be terminated.

[0131] The purpose of multiple iterative training is to increase the similarity of positive sample vectors in a group of training samples while reducing the similarity between positive sample vectors and negative sample vectors.

[0132] refer to Figure 5 , Figure 5 A flow chart of the comparative learning training phase in the embodiment of the present disclosure is shown, including steps S501 to S505:

[0133] In step S501, each positive sample audio in each group of training samples is obtained;

[0134] In step S502, each positive sample audio passes through a filter bank, a convolutional neural network and an audio encoder to obtain each positive sample vector;

[0135] In step S503, each negative sample audio in each group of training samples is obtained;

[0136] In step S504, each negative sample audio passes through a filter bank, a convolutional neural network, and an audio encoder to obtain each negative sample vector;

[0137] In step S505, contrast training is performed to make the inner product between the positive sample vector and the negative sample vector as small as possible, while the inner product between the positive sample vectors is as large as possible.

[0138] Next reference Figure 1In step S140, when the contrast loss value meets the first preset convergence condition, the feature extraction network and the audio encoder are frozen, and the classifier of the initial audio recognition model is iteratively trained according to the audio vector corresponding to each group of training samples to obtain a trained audio recognition model.

[0139] In this step, referring to the explanation of the above step S130, it can be seen that when the contrast loss value corresponding to a certain group of training samples meets the first preset convergence condition, the above feature extraction network and audio encoder can be frozen, that is, the network parameters of the feature extraction network and the network parameters of the audio encoder are kept no longer changed.

[0140] After that, the model fine-tuning stage can be entered. Specifically, the classifier of the initial audio recognition model can be iteratively trained according to the audio vectors corresponding to the above-mentioned multiple groups of training samples, that is, the network parameters of the classifier can be iteratively updated to obtain a trained audio recognition model.

[0141] It should be noted that after entering the model fine-tuning stage, given that the network parameters of the feature extraction network and the network parameters of the audio encoder have been trained at this time, in an optional implementation, each spectral information corresponding to each audio contained in each group of training samples can also be re-input into the currently trained audio recognition model, so as to re-output each audio vector corresponding to each audio contained in each group of training samples based on the feature extraction network and audio encoder with fixed parameters.

[0142] refer to Figure 6 , Figure 6 A flowchart showing how to iteratively train the classifier of the initial audio recognition model according to the audio vectors corresponding to the multiple sets of training samples to obtain the trained audio recognition model in the embodiment of the present disclosure includes steps S601 to S603:

[0143] In step S601, audio recognition processing is performed on each audio vector corresponding to each audio in each group of training samples through the classifier of the initial audio recognition model to obtain a predicted label corresponding to each audio.

[0144] In this step, the classifier of the initial audio recognition model can perform audio recognition processing on each audio vector corresponding to each audio in each group of training samples to obtain a prediction label corresponding to each audio. The prediction label represents the identification information of whether the audio predicted by the model is a forged audio.

[0145] In step S602, the true label corresponding to each audio is obtained, the classification loss value corresponding to each group of training samples is determined according to the difference value between the true label and the predicted label corresponding to each audio, and the network parameters of the classifier are updated according to the classification loss value.

[0146] In this step, the real label corresponding to each audio can be obtained, and the real label is the annotation information of whether the audio is a forged audio.

[0147] After that, the classification loss value corresponding to each group of training samples can be determined according to the difference between the true label and the predicted label corresponding to each audio. Specifically, the classification loss value L corresponding to each group of training samples can be calculated based on the following formula 2: b :

[0148]

[0149] Among them, L b Represents the classification loss value, N = 2n; y i Represents the true label corresponding to each audio. Represents the predicted label corresponding to each audio.

[0150] It should be noted that in the actual model training process, the audio vectors corresponding to a group of training samples can be input each time, and then the classification loss value corresponding to the group of training samples can be calculated based on the difference between the output of the model and the true label, and the network parameters of the classifier can be updated according to the classification loss value. After that, the audio vectors corresponding to the next group of training samples can be input, and then the next classification loss value corresponding to the next group of training samples can be calculated again based on the difference between the output of the model and the true label, and the network parameters of the classifier can be updated again according to the next classification loss value, so as to perform multiple iterative training on the classifier of the above-mentioned initial audio recognition model until a certain classification loss value obtained meets the second preset convergence condition (for example: less than or equal to the preset loss value), and the iterative training process can be terminated.

[0151] In step S603, when the classification loss value satisfies a second preset convergence condition, a trained audio recognition model is obtained.

[0152] In this step, referring to the explanation of step S602 above, when the classification loss value corresponding to a group of training samples meets the second preset convergence condition, the network parameters of the classifier can be frozen to obtain a trained audio recognition model. By minimizing the classification loss value, the classification performance of the model can be optimized, thereby improving the model's ability to identify audio.

[0153] refer to Figure 7 , Figure 7 A schematic diagram of the process of the model fine-tuning stage in the embodiment of the present disclosure is shown, including steps S701 to S704:

[0154] In step S701, the network parameters of the convolutional neural network and the audio encoder are fixed;

[0155] In step S702, each spectrum information corresponding to each audio in each group of training samples is input into the model to obtain an audio vector;

[0156] In step S703, the classifier is iteratively trained according to the degree of difference between the audio vectors corresponding to each group of training samples, so that the classification loss value is minimized;

[0157] In step S704, the network parameters of the classifier are obtained.

[0158] After obtaining the trained audio recognition model, the model application stage can be entered. Specifically, the present disclosure can obtain the audio to be recognized (which may be unknown audio that is uncertain whether it is forged audio), and use the Mel filter group to collect the frequency spectrum information to be recognized corresponding to the audio to be recognized. After that, the frequency spectrum information to be recognized can be input into the above-mentioned trained audio recognition model, and the trained audio recognition model outputs the audio recognition result, which can be used to characterize whether the audio to be recognized is forged audio.

[0159] refer to Figure 8 , Figure 8 A flow chart of the model application stage in the embodiment of the present disclosure is shown, including steps S801 and S802:

[0160] In step S801, unknown audio is input into the trained audio recognition model;

[0161] In step S802, a label of the unknown audio is obtained according to the output of the model, and the label represents whether the audio is real audio or forged audio.

[0162] refer to Fig. 9 , Fig. 9 The overall schematic diagram of the audio recognition model training method in the embodiment of the present disclosure is shown as follows: Fig. 9 As shown:

[0163] First, positive sample audio (including original audio and enhanced audio) and negative sample audio can be collected, and then each spectrum information can be collected through a Mel filter bank. Then, each audio vector can be output through a feature extraction network and an audio encoder, and the inner product between positive sample vectors can be made as large as possible and the inner product between positive and negative samples can be made as small as possible through contrast learning, so as to update the network parameters of the feature extraction network and the audio encoder;

[0164] Secondly, the network parameters of the feature extraction network and the audio encoder can be fixed, and the classifier of the initial audio recognition model can be iteratively trained using the audio vectors corresponding to multiple sets of training samples to obtain a trained audio recognition model ( Fig. 9 not shown);

[0165] Afterwards, the unknown audio can be input into the trained audio recognition model, and the recognition result (i.e., real audio or fake audio) can be output through the classifier.

[0166] Based on the above technical solutions, the present disclosure has at least the following technical effects:

[0167] First, efficient voice forgery recognition capability: By introducing contrastive learning, the present disclosure significantly improves the accuracy of voice forgery recognition, especially in the scenario of high-fidelity forged audio;

[0168] Second, strong generalization ability: The model is trained on forged audios from a variety of audio synthesis technologies (such as GPT-SoVits, CosyVoice, etc.) and combined with a variety of data augmentation techniques, giving it excellent generalization ability and the ability to cope with different types of audio forgeries;

[0169] Third, the application scope is wide: the present disclosure can not only be applied to the identification and prevention of voice fraud, but also can be used for voice authenticity verification in the fields of news, entertainment, and justice, thereby improving the overall security of society;

[0170] Fourth, technical innovation: By introducing a variety of audio processing and contrastive learning strategies, this disclosure fills the gap in current audio forgery identification technology and provides important technical support for future audio security.

[0171] The present disclosure also provides a training device for an audio recognition model. Fig.10 A schematic diagram showing the structure of a training device for an audio recognition model in an exemplary embodiment of the present disclosure is shown; Fig.10 As shown, the training device 1000 for the audio recognition model may include a sample acquisition module 1010, a feature extraction and encoding module 1020, a first training module 1030 and a second training module 1040. Among them:

[0172] The sample acquisition module 1010 is used to acquire multiple groups of training samples and collect each spectrum information corresponding to each audio contained in each group of training samples; each group of training samples includes positive sample audio and negative sample audio, the positive sample audio includes original audio and n-1 enhanced audios, and the negative sample audio includes n forged audios;

[0173] A feature extraction and encoding module 1020 is used to extract each spectrum feature corresponding to each spectrum information through a feature extraction network of the initial audio recognition model, and encode each spectrum feature through an audio encoder of the initial audio recognition model to obtain each audio vector;

[0174] A first training module 1030 is used to determine a contrast loss value corresponding to each group of training samples according to the degree of difference between the audio vectors corresponding to each group of training samples, and iteratively train the feature extraction network and the audio encoder according to the contrast loss value;

[0175] The second training module 1040 is used to freeze the feature extraction network and the audio encoder when the contrast loss value meets the first preset convergence condition, and iteratively train the classifier of the initial audio recognition model according to the audio vector corresponding to each group of training samples to obtain a trained audio recognition model.

[0176] In an exemplary embodiment of the present disclosure, each group of training samples is obtained by:

[0177] Collecting each original audio from an audio resource library;

[0178] Performing audio enhancement processing on each of the original audios to obtain n-1 enhanced audios corresponding to each of the original audios;

[0179] Audio synthesis processing is performed on each original audio through multiple audio synthesis models to obtain n forged audios corresponding to each original audio.

[0180] In an exemplary embodiment of the present disclosure, the sample acquisition module 1010 collects each spectrum information corresponding to each audio contained in each group of training samples, including:

[0181] Performing frame processing on each audio to obtain multiple audio frames;

[0182] Perform Fourier transform on each audio frame to obtain each frequency domain signal;

[0183] Input the obtained multiple frequency domain signals into the Mel filter to obtain the Mel spectrum;

[0184] Perform logarithmic transformation on the Mel spectrum to obtain each spectrum information corresponding to each audio.

[0185] In an exemplary embodiment of the present disclosure, the audio vector corresponding to each group of training samples includes a positive sample vector corresponding to each positive sample audio and a negative sample vector corresponding to each negative sample audio;

[0186] The first training module 1030 determines the contrast loss value corresponding to each group of training samples according to the difference between the audio vectors corresponding to each group of training samples, including:

[0187] Determine the contrast loss value corresponding to each group of training samples according to the inner product between each positive sample vector and the transposed vector of the target positive sample vector, the inner product between each positive sample vector and the transposed vector of each negative sample vector, the inner product between each negative sample vector and the transposed vector of the target negative sample vector, and the inner product between each negative sample vector and the transposed vector of each positive sample vector;

[0188] The target positive sample vector is randomly selected from n-1 positive sample vectors other than the positive sample vector, and the target negative sample vector is randomly selected from n-1 negative sample vectors other than the negative sample vector.

[0189] In an exemplary embodiment of the present disclosure, the first training module 1030 determines the contrast loss value corresponding to each group of training samples according to the inner product between each positive sample vector and the transposed vector of the target positive sample vector, the inner product between each positive sample vector and the transposed vector of each negative sample vector, the inner product between each negative sample vector and the transposed vector of the target negative sample vector, and the inner product between each negative sample vector and the transposed vector of each positive sample vector, including:

[0190] The contrast loss value is determined based on the following formula:

[0191]

[0192] Among them, L a represents the contrast loss value, t represents a preset temperature coefficient, and n represents the number of the positive sample audio or the negative sample audio in each group of training samples;

[0193] r i Represents each positive sample vector, r′ z A transposed vector representing the target positive sample vector;

[0194] f i Represents each negative sample vector, f′ z represents the transposed vector of the target negative sample vector.

[0195] In an exemplary embodiment of the present disclosure, the second training module 1040 iteratively trains the classifier of the initial audio recognition model according to the audio vectors corresponding to the multiple groups of training samples to obtain a trained audio recognition model, including:

[0196] Performing audio recognition processing on each audio vector corresponding to each audio in each group of training samples by using the classifier of the initial audio recognition model to obtain a predicted label corresponding to each audio;

[0197] Obtaining a true label corresponding to each audio, determining a classification loss value corresponding to each group of training samples according to a difference value between the true label corresponding to each audio and the predicted label, and updating a network parameter of the classifier according to the classification loss value;

[0198] When the classification loss value satisfies a second preset convergence condition, a trained audio recognition model is obtained.

[0199] In an exemplary embodiment of the present disclosure, the second training module 1040 determines the classification loss value corresponding to each group of training samples according to the difference value between the real label corresponding to each audio and the predicted label, including:

[0200] The classification loss value corresponding to each group of training samples is calculated based on the following formula:

[0201]

[0202] Among them, L b represents the classification loss value, N = 2n; y i represents the true label corresponding to each audio, Indicates the predicted label corresponding to each audio.

[0203] In an exemplary embodiment of the present disclosure, after obtaining the trained audio recognition model, the second training module 1040 is configured to:

[0204] Acquire the audio to be recognized, and collect the frequency spectrum information to be recognized corresponding to the audio to be recognized;

[0205] Inputting the frequency spectrum information to be recognized into the trained audio recognition model, and outputting the audio recognition result through the trained audio recognition model;

[0206] The audio recognition result is used to indicate whether the audio to be recognized is forged audio.

[0207] The specific details of each module in the above-mentioned audio recognition model training device have been described in detail in the corresponding audio recognition model training method, so they will not be repeated here.

[0208] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0209] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0210] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0211] The present disclosure also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or may exist independently without being assembled into the electronic device.

[0212] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0213] Computer-readable storage media can send, propagate or transmit programs for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0214] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiments.

[0215] In addition, an electronic device capable of implementing the above method is also provided in an embodiment of the present disclosure.

[0216] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0217] Refer to the following Fig.11 1100 according to this embodiment of the present disclosure is described. Fig.11 The electronic device 1100 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0218] like Fig.11 As shown, the electronic device 1100 is in the form of a general computing device. The components of the electronic device 1100 may include, but are not limited to: at least one processor 1110, at least one memory 1120, a bus 1130 connecting different system components (including the memory 1120 and the processor 1110), and a display 1140.

[0219] The memory stores program codes, which can be executed by the processor 1110, so that the processor 1110 executes the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processor 1110 can execute the following steps: Figure 1 As shown in: Step S110, obtaining multiple groups of training samples, and collecting each spectrum information corresponding to each audio contained in each group of training samples; each group of training samples contains positive sample audio and negative sample audio, the positive sample audio contains original audio and n-1 enhanced audios, and the negative sample audio contains n forged audios; Step S120, extracting each spectrum feature corresponding to each spectrum information through the feature extraction network of the initial audio recognition model, encoding each spectrum feature through the audio encoder of the initial audio recognition model, and obtaining each audio vector; Step S130, determining the contrast loss value corresponding to each group of training samples according to the degree of difference between the audio vectors corresponding to each group of training samples, and iteratively training the feature extraction network and the audio encoder according to the contrast loss value; Step S140, when the contrast loss value meets the first preset convergence condition, freezing the feature extraction network and the audio encoder, and iteratively training the classifier of the initial audio recognition model according to the audio vector corresponding to each group of training samples, to obtain a trained audio recognition model.

[0220] The memory 1120 may include a readable medium in the form of a volatile storage, such as a random access memory (RAM) 11201 and / or a cache memory 11202 , and may further include a read-only memory (ROM) 11203 .

[0221] Memory 1120 may also include a program / utility 11204 having a set (at least one) of program modules 11205, such program modules 11205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0222] The bus 1130 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures.

[0223] The electronic device 1100 may also communicate with one or more external devices 1200 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1100, and / or communicate with any device that enables the electronic device 1100 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1150. Furthermore, the electronic device 1100 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1160. As shown, the network adapter 1160 communicates with other modules of the electronic device 1100 via a bus 1130. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1100, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0224] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

Claims

1. A method for training an audio recognition model, characterized in that: include: Acquire multiple groups of training samples, and collect each spectrum information corresponding to each audio contained in each group of training samples; each group of training samples contains positive sample audio and negative sample audio, the positive sample audio contains original audio and n-1 enhanced audio, and the negative sample audio contains n forged audio, where n is an integer greater than 1; Extracting each spectral feature corresponding to each spectral information through a feature extraction network of the initial audio recognition model, encoding each spectral feature through an audio encoder of the initial audio recognition model to obtain each audio vector; Determining a contrast loss value corresponding to each group of training samples according to the degree of difference between the audio vectors corresponding to each group of training samples, and iteratively training the feature extraction network and the audio encoder according to multiple groups of the contrast loss values; When the contrast loss value meets a first preset convergence condition, the feature extraction network and the audio encoder are frozen, and the classifier of the initial audio recognition model is iteratively trained according to the audio vectors corresponding to the multiple groups of training samples to obtain a trained audio recognition model.

2. The method according to claim 1, characterized in that Each group of training samples is obtained in the following way: Collecting each original audio from an audio resource library; Performing audio enhancement processing on each of the original audios to obtain n-1 enhanced audios corresponding to each of the original audios; Audio synthesis processing is performed on each original audio through multiple audio synthesis models to obtain n forged audios corresponding to each original audio.

3. The method according to claim 1, characterized in that The collecting of each spectrum information corresponding to each audio contained in each group of training samples includes: Performing frame processing on each audio to obtain multiple audio frames; Perform Fourier transform on each audio frame to obtain each frequency domain signal; Input the obtained multiple frequency domain signals into the Mel filter to obtain the Mel spectrum; Perform logarithmic transformation on the Mel spectrum to obtain each spectrum information corresponding to each audio.

4. The method according to claim 1, characterized in that: The audio vector corresponding to each group of training samples includes a positive sample vector corresponding to each positive sample audio and a negative sample vector corresponding to each negative sample audio; The determining, according to the degree of difference between the audio vectors corresponding to each group of training samples, the contrast loss value corresponding to each group of training samples comprises: Determine the contrast loss value corresponding to each group of training samples according to the inner product between each positive sample vector and the transposed vector of the target positive sample vector, the inner product between each positive sample vector and the transposed vector of each negative sample vector, the inner product between each negative sample vector and the transposed vector of the target negative sample vector, and the inner product between each negative sample vector and the transposed vector of each positive sample vector; The target positive sample vector is randomly selected from n-1 positive sample vectors other than the positive sample vector, and the target negative sample vector is randomly selected from n-1 negative sample vectors other than the negative sample vector.

5. The method according to claim 4, characterized in that Determining the contrast loss value corresponding to each group of training samples according to the inner product between each positive sample vector and the transposed vector of the target positive sample vector, the inner product between each positive sample vector and the transposed vector of each negative sample vector, the inner product between each negative sample vector and the transposed vector of the target negative sample vector, and the inner product between each negative sample vector and the transposed vector of each positive sample vector, include: The contrast loss value is determined based on the following formula: Among them, L a represents the contrast loss value, t represents a preset temperature coefficient, and n represents the number of the positive sample audio or the negative sample audio in each group of training samples; r i Represents each positive sample vector, r z ′ represents the transposed vector of the target positive sample vector; f i Represents each negative sample vector, f z ′ represents the transposed vector of the target negative sample vector.

6. The method according to claim 1, characterized in that The iterative training of the classifier of the initial audio recognition model according to the audio vectors corresponding to the multiple groups of training samples to obtain a trained audio recognition model includes: Performing audio recognition processing on each audio vector corresponding to each audio in each group of training samples by using the classifier of the initial audio recognition model to obtain a predicted label corresponding to each audio; Obtaining a true label corresponding to each audio, determining a classification loss value corresponding to each group of training samples according to a difference value between the true label corresponding to each audio and the predicted label, and updating a network parameter of the classifier according to the classification loss value; When the classification loss value satisfies a second preset convergence condition, a trained audio recognition model is obtained.

7. The method according to claim 6, characterized in that The determining the classification loss value corresponding to each group of training samples according to the difference value between the real label corresponding to each audio and the predicted label includes: The classification loss value corresponding to each group of training samples is calculated based on the following formula: Among them, L b represents the classification loss value, N = 2n; y i represents the true label corresponding to each audio, Indicates the predicted label corresponding to each audio.

8. The method according to any one of claims 1 to 7, characterized in that: After obtaining the trained audio recognition model, the method further includes: Acquire the audio to be recognized, and collect the frequency spectrum information to be recognized corresponding to the audio to be recognized; Inputting the frequency spectrum information to be recognized into the trained audio recognition model, and outputting the audio recognition result through the trained audio recognition model; The audio recognition result is used to indicate whether the audio to be recognized is forged audio.

9. A training device for an audio recognition model, characterized in that: include: A sample acquisition module is used to acquire multiple groups of training samples and collect each spectrum information corresponding to each audio contained in each group of training samples; each group of training samples includes positive sample audio and negative sample audio, the positive sample audio includes original audio and n-1 enhanced audio, and the negative sample audio includes n forged audio; A feature extraction and encoding module, used to extract each spectrum feature corresponding to each spectrum information through a feature extraction network of the initial audio recognition model, and encode each spectrum feature through an audio encoder of the initial audio recognition model to obtain each audio vector; A first training module, configured to determine a contrast loss value corresponding to each group of training samples according to a degree of difference between audio vectors corresponding to each group of training samples, and iteratively train the feature extraction network and the audio encoder according to the contrast loss value; The second training module is used to freeze the feature extraction network and the audio encoder when the contrast loss value meets the first preset convergence condition, and iteratively train the classifier of the initial audio recognition model according to the audio vector corresponding to each group of training samples to obtain a trained audio recognition model.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the training method of the audio recognition model described in any one of claims 1 to 8 is implemented.

11. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the audio recognition model training method according to any one of claims 1 to 8 by executing the executable instructions.

Images