Audio forgery detection method and system based on audio latent feature contrast learning

Abstract

The present invention discloses an audio forgery detection method and system based on audio latent feature contrast learning, belonging to the technical field of audio forgery detection. The present invention first performs data enhancement on original audio data to generate a forged audio data set, and then constructs an audio detection model, which includes a contrast learning model. The audio detection model is then trained in the first stage based on the forged audio data set. After the first stage training is completed, the contrast learning model is used to perform second stage training. Finally, audio forgery detection is performed based on the audio detection model that has completed the first and second stage training, thereby significantly improving detection accuracy and generalization ability.

Description

Technical Field

[0001] The present invention belongs to the technical field of audio forgery detection, and in particular relates to an audio forgery detection method and system based on audio latent feature comparative learning. Background Art

[0002] With the rapid development of audio information services, the user base continues to grow. Currently, the number of online music users in my country has reached 608 million. In particular, with the application of new AI technologies and applications such as generative artificial intelligence (AIGC) in the audio field, the output of deep learning-based audio generation and cloning algorithms is increasingly close to real audio, leading to the further accumulation and amplification of legal risks in the process of audio dissemination. Therefore, the legal use of audio data is currently a matter of great concern to society.

[0003] Current approaches to audio forgery detection primarily include: those based on audio signal features, such as phase spectrograms, mel-spectrograms, spectrograms, and improved time delays; and those based on machine learning, such as linear SVMs, weighted K-nearest neighbor methods, and boosted tree ensembles. However, these technologies still suffer from limitations such as insufficient detection accuracy and weak generalization. Specifically, methods based on audio signal features, such as phase spectrograms and mel-spectrograms, fail to fully capture the complex variations in audio forgeries, making it difficult to distinguish genuine audio from fakes when faced with advanced forgery techniques. Furthermore, when audio is presented in complex environments, ambient noise can significantly interfere with feature extraction, resulting in reduced accuracy. Furthermore, these methods are insensitive to emerging forgery patterns and struggle to adapt quickly. Machine learning-based models rely heavily on the quality and diversity of training data. Incomplete samples or mislabeling can lead to numerous false positives, resulting in poor detection performance for forged audio in rare and unique scenarios. Furthermore, their generalization is insufficient, making them incapable of adapting to evolving new forgery techniques. They consume significant computational resources, making them difficult to apply in resource-constrained scenarios, and are vulnerable to adversarial attacks, which can invalidate detection results. Therefore need a method to solve the above problems badly. Summary of the Invention

[0004] To solve the above technical problems, the present invention proposes an audio forgery detection method and system based on audio latent feature contrast learning to solve the problems existing in the above-mentioned prior art.

[0005] In a first aspect, to achieve the above-mentioned objectives, the present invention provides an audio forgery detection method based on audio latent feature contrast learning, comprising the following steps:

[0006] Perform data enhancement on the original audio data to generate a fake audio dataset;

[0007] Constructing an audio detection model, wherein the audio detection model includes a contrastive learning model;

[0008] Performing a first-stage training on the audio detection model based on the forged audio dataset;

[0009] After completing the first stage of training, the contrastive learning model is used for the second stage of training;

[0010] The audio is forged detected based on the audio detection model that has completed the first stage training and the second stage training.

[0011] Optionally, the process of performing data augmentation on the original audio data to generate a fake audio dataset includes:

[0012] Confirm the distribution ratio of positive and negative samples in the data set. If the positive and negative sample ratio is not equal to 1:1, adjust the data;

[0013] The adjusted original audio data is then augmented to generate a fake audio dataset.

[0014] Optionally, the process of performing data enhancement on the adjusted original audio data includes: performing Gaussian noise enhancement, waveform shifting, waveform stretching, and pitch correction on the adjusted original audio data.

[0015] Optionally, an audio detection model is constructed, wherein the audio detection model further includes: a Sinc layer, a residual block, a GRU layer, and a fully connected layer.

[0016] Optionally, the audio detection model is trained in a first phase based on the forged audio dataset. The number of training rounds in the first phase is N. The first phase training ends after the number of training rounds is completed. The first phase training process includes:

[0017] The model is trained based on the cross entropy loss function.

[0018] Optionally, after completing the first phase of training, the second phase of training using the contrastive learning model includes:

[0019] The second stage of training is performed based on the cross entropy loss function and the contrastive learning loss function.

[0020] In a second aspect, the present invention further provides an audio forgery detection system based on audio latent feature contrast learning, which is used to implement an audio forgery detection method based on audio latent feature contrast learning. The system comprises:

[0021] The data processing module is used to perform data enhancement on the original audio data and generate a fake audio data set;

[0022] A model building module is used to build an audio detection model, wherein the audio detection model includes a contrastive learning model, a Sinc layer, a residual block, a GRU layer, and a fully connected layer;

[0023] a model training module, configured to perform a first-stage training of the audio detection model based on the forged audio dataset, and after completing the first-stage training, perform a second-stage training using a contrastive learning model;

[0024] A detection module is used to detect audio forgery based on the audio detection model that has completed the first stage training and the second stage training.

[0025] Optionally, the data processing module includes:

[0026] The data enhancement unit is used to perform Gaussian noise enhancement, waveform displacement, waveform stretching and pitch correction on the original audio data.

[0027] In a third aspect, the present invention further provides a computer terminal device, comprising:

[0028] one or more processors;

[0029] a memory, coupled to the processor, for storing one or more programs;

[0030] When the one or more programs are executed by the one or more processors, the one or more processors implement, for example, an audio forgery detection method based on comparative learning of audio latent features.

[0031] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements, for example, an audio forgery detection method based on comparative learning of audio latent features.

[0032] Compared with the prior art, the present invention has the following advantages and technical effects:

[0033] The present invention provides a method and system for detecting audio forgeries based on contrastive learning of audio latent features. First, the system generates a forged audio dataset covering complex scenarios by performing multi-dimensional data augmentation on the original data (including Gaussian noise addition, waveform shifting, stretching, and pitch correction). Secondly, the system constructs a RawNet2-C audio detection model that integrates Sinc convolutional layers, residual blocks, and feature scaling mapping, and integrates a contrastive learning module. After the model undergoes a first phase of training based on the augmented data, it further optimizes its classification and feature discrimination capabilities through a two-stage training strategy, ultimately significantly improving the model's detection accuracy for highly realistic forged audio. Through data augmentation and phased training, the model effectively enhances its adaptability to background noise and changes in speech rate and pitch. It also directly extracts deep latent features from the original waveform, avoiding the limitations of traditional manual feature design and enhancing robustness and discrimination capabilities in complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0035] Figure 1 A schematic diagram of audio enhancement according to an embodiment of the present invention;

[0036] Figure 2 Schematic diagram of residual connection according to an embodiment of the present invention;

[0037] Figure 3 This is a schematic diagram of the FMS feature scaling according to an embodiment of the present invention;

[0038] Figure 4 Flowchart of a method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0040] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0041] Example 1

[0042] like Figure 4 As shown, this embodiment provides an audio forgery detection method based on audio latent feature contrast learning, including:

[0043] Perform data enhancement on the original audio data to generate a fake audio dataset;

[0044] Constructing an audio detection model, wherein the audio detection model includes a contrastive learning model;

[0045] Performing a first-stage training on the audio detection model based on the forged audio dataset;

[0046] After completing the first stage of training, the contrastive learning model is used for the second stage of training;

[0047] The audio is forged detected based on the audio detection model that has completed the first stage training and the second stage training.

[0048] Specifically, the above process includes:

[0049] Step 1: Generate enhanced data based on the original audio data and data augmentation methods to build a fake audio dataset.

[0050] Step 2: Build the model RawNet2-C based on audio latent features combined with contrastive learning.

[0051] Step 3: Perform the first phase of training on RawNet2-C based on the enhanced dataset to initially achieve forged audio detection.

[0052] Step 4: After the first phase of training of the RawNet2-C model is completed, contrastive learning is integrated and based on the mechanism of contrastive learning, the second phase of training is carried out to optimize and improve the detection effect of the model.

[0053] As an implementation method of this embodiment, the process of performing data enhancement on the original audio data to generate a forged audio dataset includes:

[0054] Confirm the distribution ratio of positive and negative samples in the data set. If the positive and negative sample ratio is not equal to 1:1, adjust the data;

[0055] The adjusted original audio data is subjected to data enhancement to generate a forged audio dataset, wherein the forged audio dataset includes real and forged audio.

[0056] Specifically, the process of step 1 is as follows:

[0057] Confirm the distribution ratio of positive and negative samples in the data set. If the distribution ratio of positive and negative samples is not equal to 1:1, adjust the data;

[0058] Use the enhancement method to perform data enhancement on the audio signal. The audio enhancement results are as follows: Figure 1 shown.

[0059] As an implementation method in this embodiment, the process of performing data enhancement on the original audio data after adjustment includes: performing Gaussian noise enhancement, waveform displacement, waveform stretching and pitch correction on the original audio data after adjustment.

[0060] Specifically, the enhanced methods include:

[0061] (1) Gaussian noise enhancement: Add Gaussian noise with a mean of 0 and an adjustable standard deviation to the original audio signal to simulate the background noise in the real environment.

[0062] (2) Waveform displacement: The audio signal is shifted on the time axis without changing the frequency content of the signal, simulating slight changes in the signal in time, such as slight delay or advance of the speaker.

[0063] (3) Waveform stretching: This method adjusts the duration of a signal by changing the time interval between each sample point in the signal, thereby changing the playback speed of the audio signal without changing its pitch, and is used to simulate different speaking speeds.

[0064] (4) Pitch correction: The pitch is adjusted by changing the fundamental frequency of the signal while keeping the relative positions of other frequency components unchanged. The pitch of the audio signal is changed without changing its playback speed, which is used to simulate speech with different tones.

[0065] As an implementation method of this embodiment, an audio detection model is constructed, which further includes: a Sinc layer, a residual block, a GRU layer and a fully connected layer, wherein the GRU layer is a gated recurrent unit (GRU) layer.

[0066] Specifically, the process of step 2 is as follows:

[0067] First, Sinc convolution directly processes audio time series signals. Sinc convolution is an interpretable convolutional filter structure that uses parameterized bandpass filter characteristics to extract frame rate information directly from raw waveforms within a deep learning framework. This structure, located in the first layer of the network, offers an alternative to traditional spectrum extraction methods such as MFCC, perceptual linear prediction, and filter bank extraction.

[0068] For audio signals, Sinc convolution uses a predefined function g to extract audio signal features, where g contains only a few learnable variables. , the definition of convolution operation based on Sinc function is as follows:

[0069] (1)

[0070] in is the voice signal, n is the length of the voice data, is the output of the filter. The g function is defined as a rectangular bandpass filter, and its frequency domain characteristics are as follows:

[0071] (2)

[0072] Among them, f represents the different frequency components in the audio signal. and are low and high cutoff frequencies respectively, both of which are learnable parameters. is a rectangular function in the amplitude-frequency domain and is defined as follows:

[0073] (3)

[0074] After Fourier transform, the time domain expression is as follows:

[0075] (4)

[0076] in The function is defined as .

[0077] Finally, in order to smooth the truncation characteristics of the g function, a window function is multiplied on the basis of the g function. Here, the Hamming window is used, which has the following form:

[0078] (5)

[0079] (6)

[0080] represents the new function obtained after multiplying the Hamming window function, represents the Hamming window function, Represents the length of the Hamming window function, that is, the number of samples covered by the window, and the input features are further processed using the residual block to extract frame-level features. The core idea of ​​the residual block is to introduce a skip connection, which allows the signal in the network to bypass one or more layers, such as Figure 2 shown.

[0081] In its most basic form, a residual block can be expressed as:

[0082] (7)

[0083] in, It is the audio feature extracted after the convolution operation. is the residual function, which represents the residual mapping that needs to be learned, represents the set of learnable parameters in the residual function, and c is the output feature.

[0084] Residual blocks require the network to learn only the difference between input and output, rather than the full output. This setup simplifies the learning task, as learning the difference is often easier than learning the full output target. Furthermore, skip connections allow input features to be passed directly to subsequent layers, which not only facilitates gradient flow but also ensures that feature information is efficiently transferred throughout the network, avoiding the gradual degradation of features in deep networks.

[0085] Use feature scaling mapping to improve the expressiveness of the residual block output features. First, let is the output feature of the residual block, that is ,in is the length of the sequence in time, is the number of filters. First, global average pooling is performed on the time axis, then feedforward through the fully connected layer, and then activated by Sigmoid to derive the scale vector for performing feature scaling mapping FMS. By expressing the scale vector as , and then designed a scaled feature , feature scaling such as Figure 3 As shown, the features are scaled by the following additive feature:

[0086] (8)

[0087] in, represents the output features of the residual block, represents the feature after additive feature scaling, represents the corresponding element in the scale vector used for feature scaling, . It is also possible to perform multiplicative scaling on features:

[0088] (9)

[0089] The two can also be executed interactively in sequence, as shown below:

[0090] (10)

[0091] (11)

[0092] Contrastive learning is used to maximize the similarity between the features of real audio data and the features of forged audio data, while minimizing the similarity between features of the same type of data, that is, between the features of real sample data or between the features of forged audio data.

[0093] Model construction: The RawNet2-C model first uses Sinc convolution to directly process the temporal signal features of audio. It then uses residual blocks to further process the input features to extract frame-level features. Feature scaling mapping is then used to improve the expressiveness of the residual block output features. Finally, contrastive learning is used to maximize the similarity between the features of real and forged audio data to improve the model's discriminative ability. The model structure is shown in Table 1.

[0094] Table 1 RawNet2-C model structure

[0095] layer Input 64,000 samples Output shape SincNet Layer Convolution (129, 1, 128) Max Pooling (3) Normalization and LeakyReLU (21290,128) Residual block ×2 Normalization and LeakyReLU convolution (3, 1, 128) Normalization and LeakyReLU convolution (3, 1, 128) Max pooling (3) FMS (2365,128) Residual block ×4 Normalization and LeakyReLU convolution (3, 1, 512) Normalization and LeakyReLU convolution (3, 1, 512) Max pooling (3) FMS (29,512) GRU GRU(1024) (1024) Fully connected layer 1024 (1024) Output 1024 4

[0096] In Table 1, the functions at each level and the meaning of the function parameters are explained as follows:

[0097] For the convolution (129, 1, 128) in the SincNet convolutional layer:

[0098] This is a one-dimensional convolution operation. The convolution kernel size is 129 (i.e., its length on the time axis is 129), the number of input channels is 1 (assuming the input data is a one-dimensional signal, such as a speech signal), and the number of output channels is 128. It is primarily used to extract local features of the input signal by sliding the convolution kernel over the input signal to calculate these features.

[0099] For the maximum pooling (3) in the SincNet convolutional layer:

[0100] Max pooling is a downsampling operation. Here, the pooling window size is 3, meaning the maximum value is taken every 3 units. It reduces the dimensionality of the data while retaining the maximum eigenvalue within the local region, helping to reduce computational complexity and preventing overfitting to a certain extent.

[0101] For the normalization and LeakyReLU in the SincNet convolutional layer:

[0102] Normalization (such as batch normalization) can speed up the network training process by adjusting and scaling the input of neurons so that the input of each layer has zero mean and unit variance, making the network training more stable.

[0103] LeakyReLU is an activation function that solves the problem of the traditional ReLU function having a zero gradient when the input is negative. When the input is negative, LeakyReLU outputs a very small value (determined by the negative slope parameter), allowing the neuron to have a small gradient even when the input is negative, thus preventing the neuron from "dying".

[0104] For the normalization and LeakyReLU of the residual block ×2:

[0105] The function is the same as above, and the input is normalized and activated at the beginning of the residual block.

[0106] For the convolution (3, 1, 128) in the residual block × 2:

[0107] The convolution kernel size is 3, the number of input channels is 1, and the number of output channels is 128. Continue to extract and transform features.

[0108] For the normalization and LeakyReLU of the residual block ×2:

[0109] Map and normalize the features obtained after convolution.

[0110] For the convolution (3, 1, 128) in the residual block × 2:

[0111] Perform convolution operation again to further extract deeper features.

[0112] For the maximum pooling (3) of the residual block ×2:

[0113] Perform downsampling operations to reduce data dimensions and retain key features.

[0114] For the FMS of the residual block ×2:

[0115] Perform feature scaling.

[0116] For the normalization and LeakyReLU of the residual block × 4:

[0117] The convolutional features are mapped and normalized.

[0118] For the convolution (3, 1, 512) in the residual block × 4:

[0119] The convolution kernel size is 3, the number of input channels is 1, and the number of output channels is 512, which is used to extract more complex features. As the number of channels increases, the expressive power of the features increases.

[0120] For the normalization and LeakyReLU of the residual block × 4:

[0121] The convolutional features are mapped and normalized.

[0122] For the convolution (3, 1, 512) in the residual block × 4:

[0123] Continue the convolution operation to deepen the feature extraction level.

[0124] For the maximum pooling (3) of the residual block × 4:

[0125] Reduce the dimensionality of the data.

[0126] For the FMS of the residual block × 4:

[0127] Perform feature scaling.

[0128] For GRU (Gated Recurrent Unit, gated recurrent unit) GRU (1024):

[0129] The GRU is a recurrent neural network architecture used to process sequential data. The 1024 here represents the dimension of the GRU's hidden units. It controls the flow of information through update and reset gates, enabling it to better handle long-term dependencies in sequences and pass information from previous time steps to later ones.

[0130] For 1024 of the fully connected layers:

[0131] The fully connected layer linearly transforms the previously extracted features (dimension is 1024). Each neuron is connected to all neurons in the previous layer to map the features to a new space for further classification or regression tasks.

[0132] For the output layer:

[0133] The dimension of the output layer is 4, which may be a four-class classification problem, where each output unit corresponds to the probability of a category (if it is a classification task and uses the softmax activation function), or represents four different output features (if it is other tasks such as regression).

[0134] As an implementation method of this embodiment, the audio detection model is trained in the first phase based on the forged audio dataset. The number of training rounds in the first phase is N. The first phase training ends after the number of training rounds is completed. The first phase training process includes:

[0135] The model is trained based on the cross entropy loss function.

[0136] Specifically, the process of step 3 includes:

[0137] First, without using contrastive learning, the first stage of RawNet2-C training is performed to give the model preliminary audio authentication capabilities. The loss function in the first stage is as follows:

[0138] (12)

[0139] in, is the cross entropy loss function, y is the label, the label for real audio data is 1, the label for fake audio data is 0, and p is the predicted probability that the input audio data is real data.

[0140] As an implementation method of this embodiment, after completing the first stage of training, the process of using the contrastive learning model to perform the second stage of training includes:

[0141] The second stage of training is performed based on the cross entropy loss function and the contrastive learning loss function.

[0142] Specifically, the process of step 4 includes:

[0143] Contrastive learning was used to train the model in the second phase to improve its discrimination ability. The second phase used a speech-enhanced dataset for training. Since the model already had basic recognition capabilities after the first round of training, the number of training rounds in the second phase was relatively small, 30 rounds in total. The convergence process was relatively stable in the early stages, but would fluctuate later. Finally, the model with the best performance on the validation set was selected.

[0144] The loss function of contrastive learning is:

[0145] (13)

[0146] in, Represents the loss function of contrastive learning. The value of k depends on whether the audio data is of the same type. When it is the same type of audio data, k is 1, otherwise k is 0. and Respectively Audio data features of different categories and the same category, r is a constant, controlling the examples between similar samples, Represents the similarity function, which is used to measure the similarity between two features, and P represents the total number of samples during training.

[0147] Finally, in the second stage, the loss function of the model is as follows:

[0148] (14)

[0149] in and are weights, among which Mainly used for the model's learning of forged features. It is used to narrow the distance between samples with the same features, for example, to make samples with forged features more aggregated in the feature space to facilitate forgery classification. Therefore, it is hoped that the model can achieve better forgery detection results through contrastive learning in the second stage. Smaller, Larger, used in experiments is 0.1, When it is 0.45, the effect is better. represents the total loss function.

[0150] Data from actual experiments:

[0151] (1) Select experimental data:

[0152] In this experiment, ASVspoof2019 LA is used as the dataset. The training set contains approximately 50,000 data items, including approximately 2,500 real audio data items and approximately 48,000 forged audio data items. To meet the needs of comparative learning, this experiment augmented the real audio data. After augmentation, the real data is approximately 54,000 items, which basically meets the 1:1 sample ratio requirement.

[0153] (2) Experimental results:

[0154] This experiment was conducted on the ASVspoof2019 LA test set, which contains approximately 140,000 audio samples, including approximately 7,000 real audio data and approximately 133,000 forged audio data. The accuracy of the audio data identification was 95.56%.

[0155] Based on this, an embodiment of the present invention provides an audio forgery detection method based on contrastive learning of audio latent features. First, data enhancement is performed on the original audio data to generate a forged audio dataset. Second, an audio detection model is constructed, which includes a contrastive learning model. Then, the audio detection model is trained in the first stage based on the forged audio dataset. After the first stage training is completed, the contrastive learning model is used to perform second stage training. Finally, the audio is forged based on the audio detection model that has completed the first and second stage training. On the ASVspoof2019 LA test set, the model achieved a detection accuracy of 95.56% for 140,000 audios, significantly outperforming traditional methods. By simulating diverse noise and speech changes through data enhancement, the model can adapt to complex real-world scenarios (such as background noise and speech speed / pitch differences). Combining Sinc convolution with residual blocks, deep latent features are extracted directly from the original waveform, avoiding the limitations of artificial feature design. The two-stage training strategy effectively balances classification and feature discrimination tasks, enhancing the model's sensitivity to subtle forgery traces.

[0156] Example 2

[0157] In this embodiment, a computer terminal device is provided, including:

[0158] one or more processors;

[0159] a memory, coupled to the processor, for storing one or more programs;

[0160] When the one or more programs are executed by the one or more processors, the one or more processors implement the methods in the above embodiments.

[0161] In this embodiment, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the method in the above embodiment is implemented.

[0162] In this embodiment, an electronic device is further provided, including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to execute the method in the above embodiment.

[0163] The above program can be executed in a processor or stored in a memory (or computer-readable medium). Computer-readable media includes both permanent and non-permanent, removable and non-removable media, and can be implemented using any method or technology to store information. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0164] These computer programs can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps of the functions specified in one or more blocks can be implemented by different modules corresponding to different steps.

[0165] This embodiment provides such a device or system. The system is called an audio forgery detection system based on audio latent feature contrast learning, and includes:

[0166] The data processing module is used to perform data enhancement on the original audio data and generate a fake audio data set;

[0167] A model building module is used to build an audio detection model, wherein the audio detection model includes a contrastive learning model, a Sinc layer, a residual block, a GRU layer, and a fully connected layer;

[0168] a model training module, configured to perform a first-stage training of the audio detection model based on the forged audio dataset, and after completing the first-stage training, perform a second-stage training using a contrastive learning model;

[0169] A detection module is used to detect audio forgery based on the audio detection model that has completed the first stage training and the second stage training.

[0170] As an implementation method of this embodiment, the data processing module includes:

[0171] A data enhancement unit, used to perform Gaussian noise enhancement, waveform shifting, waveform stretching, and pitch correction on the original audio data;

[0172] The data distribution adjustment unit is used to confirm the distribution ratio of positive and negative samples in the data set and adjust the data if it is not equal to 1:1.

[0173] As an implementation method in this embodiment, when the training module performs the first-stage training on the audio detection model based on the forged audio dataset, the training is performed using a cross-entropy loss function.

[0174] As an implementation method in this embodiment, after completing the first stage of training, the training module uses the contrastive learning model to perform the second stage of training, and adopts the cross entropy loss function and the contrastive learning loss function for training.

[0175] The system or device is used to implement the functions of the method in the above-mentioned embodiment. Each module in the system or device corresponds to each step in the method, which has been explained in the method and will not be repeated here.

[0176] Through the above implementation, the problem of audio forgery detection based on contrastive learning of audio latent features in the related art is solved. By combining contrastive learning with a specific model architecture and a phased training strategy, the problem of insufficient discrimination ability of traditional audio forgery detection methods in complex scenarios is solved.

[0177] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. An audio forgery detection method based on audio latent feature contrast learning, characterized in that: The following steps are involved: Perform data enhancement on the original audio data to generate a fake audio dataset; Constructing an audio detection model, wherein the audio detection model includes a contrastive learning model; The audio detection model is trained in a first phase based on the forged audio dataset, wherein the first phase training process includes: training the model based on a cross entropy loss function; The loss function of the first stage is expressed as follows: in, is the cross entropy loss function, y is the label, the label for real audio data is 1, the label for fake audio data is 0, and p is the predicted probability that the input audio data is real data; After completing the first stage of training, the contrastive learning model is used to perform the second stage of training, wherein the second stage of training includes: performing the second stage of training based on the cross entropy loss function and the contrastive learning loss function; The loss function of contrastive learning in the second stage is: in, Represents the loss function of contrastive learning. The value of k depends on whether the audio data is of the same type. When it is the same type of audio data, k is 1, otherwise k is 0. and Respectively Audio data features of different categories and the same category, r is a constant, controlling the examples between similar samples, Represents the similarity function, which is used to measure the similarity between two features. P represents the total number of samples during training. Finally, in the second stage, the loss function of the model is as follows: in and are weights, among which Used for model learning of forged features, Used to shorten the distance between samples with the same characteristics. represents the total loss function; The audio is forged detected based on the audio detection model that has completed the first stage training and the second stage training.

2. The method according to claim 1, characterized in that The process of performing data augmentation on the original audio data and generating a fake audio dataset includes: Confirm the distribution ratio of positive and negative samples in the data set. If the positive and negative sample ratio is not equal to 1:1, adjust the data; The adjusted original audio data is then augmented to generate a fake audio dataset.

3. The method according to claim 1, characterized in that The process of performing data enhancement on the adjusted original audio data includes: performing Gaussian noise enhancement, waveform shifting, waveform stretching and pitch correction on the adjusted original audio data.

4. The method according to claim 1, wherein An audio detection model is constructed, wherein the audio detection model further includes: a Sinc layer, a residual block, a GRU layer, and a fully connected layer.

5. The method according to claim 1, wherein The audio detection model is trained in the first phase based on the forged audio data set. The number of training rounds in the first phase is N. The first phase training ends after the number of training rounds is completed.

6. An audio forgery detection system based on audio latent feature contrast learning, characterized in that: The system is used to implement the steps of the audio forgery detection method based on audio latent feature contrast learning as described in any one of claims 1 to 5, and the system includes: The data processing module is used to perform data enhancement on the original audio data and generate a fake audio data set; A model building module is used to build an audio detection model, wherein the audio detection model includes a contrastive learning model, a Sinc layer, a residual block, a GRU layer, and a fully connected layer; a model training module, configured to perform a first-stage training of the audio detection model based on the forged audio dataset, and after completing the first-stage training, perform a second-stage training using a contrastive learning model; A detection module is used to detect audio forgery based on the audio detection model that has completed the first stage training and the second stage training.

7. The system according to claim 6, characterized in that The data processing module includes: A data enhancement unit, used to perform Gaussian noise enhancement, waveform shifting, waveform stretching, and pitch correction on the original audio data; The data distribution adjustment unit is used to confirm the distribution ratio of positive and negative samples in the data set and adjust the data if it is not equal to 1:

1.

8. A computer terminal device, characterized in that: include: one or more processors; a memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the audio forgery detection method based on audio latent feature contrast learning according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the audio forgery detection method based on audio latent feature contrast learning according to any one of claims 1 to 5 is implemented.

Images