Counterfeit audio region detection method based on feature clustering and comparative learning

Abstract

The invention discloses a forged audio region detection method based on feature clustering and comparative learning, and relates to the field of audio signal processing. Comprising a multi-dimensional feature extraction module, a feature fusion module, a clustering-based frame-level pseudo tag generation module, a region coarse positioning module, a comparative learning-based region forged audio detection module, a region repositioning module and a post-processing module. According to the method, the multi-dimensional information is extracted from the audio data, the clustering method is used for generating the frame-level false label for the audio data, different types of audio areas are positioned, and the accurate false area positioning and audio authenticity detection results can be obtained by comparing, learning and training the audio area authenticity detection model; the method is not only suitable for single-language forged audio detection, but also can be popularized to multi-language and multi-accent audio scenes, is especially suitable for judicial expertise, media auditing and other application scenes needing accurate audio analysis, and has wide practical value.

Description

Technical Field

[0001] The present invention relates to the technical field of audio signal processing, and in particular to a forged audio region detection method based on feature clustering and contrast learning. Background Art

[0002] With the rapid development of audio generation technology, fake audio technology has gradually matured and has been widely used in many fields. However, fake audio not only brings entertainment and technological innovation, but also poses a serious threat to security, justice and media credibility. In recent years, deep fake technology has greatly improved the realism of fake voices, and traditional fake detection technology faces huge challenges.

[0003] Existing forged audio detection methods mainly focus on judging the authenticity of the overall audio, usually by analyzing global features to distinguish whether the audio is forged. However, most of these detection methods have the following problems: First, most methods only rely on single-dimensional features, which makes it difficult to fully capture the multi-level characteristics of forged audio, which results in less than ideal detection accuracy and robustness when dealing with different types of forged audio. Second, most technologies focus on the detection of the entire audio, which is insufficient when dealing with forgeries in which only local areas of the audio have been tampered with. The detection algorithm often tends to ignore these small but critical tampering.

[0004] In view of the above problems, the present invention proposes a forged audio region detection method based on feature clustering and contrastive learning. Summary of the invention

[0005] The purpose of the present invention is to solve the defects in the prior art and to propose a forged audio region detection method based on feature clustering and contrast learning.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A forged audio region detection method based on feature clustering and contrastive learning includes a multi-dimensional feature extraction module, a feature fusion module, a clustering-based frame-level pseudo-label generation module, a region coarse positioning module, a region forged audio detection module based on contrastive learning, a region repositioning module, and a post-processing module.

[0008] Furthermore, the multi-dimensional feature extraction module is used to extract multi-dimensional features from the audio signal;

[0009] Furthermore, the feature fusion module is connected to the multi-dimensional feature extraction module, and is used to fuse the extracted multi-dimensional features, map the high-dimensional features to the low-dimensional space through the dimensionality reduction algorithm, and reduce redundant information;

[0010] Furthermore, the clustering-based frame-level pseudo-label generation module is connected to the feature fusion module and is used to divide the audio frames into two categories using an unsupervised clustering algorithm according to the fused features to generate frame-level pseudo-labels;

[0011] Furthermore, the region coarse positioning module is connected to the clustering-based frame-level pseudo-label generation module, and is used to preliminarily classify the audio frames based on the pseudo-labels generated by clustering, and roughly locate the audio regions of different categories;

[0012] Furthermore, the regional forged audio detection module based on contrastive learning is connected to the multi-dimensional feature extraction module, the regional rough positioning module, and the regional repositioning module, and is used to perform in-depth detection on regional audios of different categories to determine the authenticity of the audios in the divided areas;

[0013] The regional forged audio detection module based on contrastive learning includes:

[0014] An audio segmentation module, connected to the region coarse positioning module and the region repositioning module, for segmenting the audio into a plurality of substrings according to the result of the region positioning;

[0015] A regional audio enhancement module, connected to the audio segmentation module, for enhancing the segmented audio substring samples through a variety of audio transformations to improve the diversity of the samples;

[0016] A positive and negative sample construction module, connected to the regional audio enhancement module, for constructing positive and negative sample pairs based on samples of different categories after audio enhancement;

[0017] A loss function design module that integrates contrast loss is used to design an effective loss function for the forged audio detection model to improve the classification accuracy of the model;

[0018] The regional forged audio detection module is connected to the positive and negative sample construction module, the loss function design module of the fusion contrast loss, and the regional relocation module, and is used to train a model for authenticity detection of regional samples of the same category, and then use the trained model to relocate the authenticity region.

[0019] Furthermore, the region relocation module is connected to the region forged audio detection module based on contrastive learning, and is used to re-detect the audio at the frame level based on the detection model optimized by contrastive learning, further refine and adjust the boundaries of the true and false regions, and improve the detection accuracy.

[0020] Furthermore, the post-processing module is connected to the regional forged audio detection module based on contrastive learning, and is used to smooth and standardize the output results of the detection model, and output accurate forged region positioning and audio authenticity detection results.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] 1. The forged audio area detection method based on frame-level multi-dimensional feature fusion and contrastive learning provided in the present invention introduces frame-level multi-dimensional feature fusion and contrastive learning technology, which can accurately detect and locate the forged area in the audio, and effectively solves the problem that local forgery is difficult to handle in the prior art;

[0023] 2. The detection method of the present invention is not only applicable to forgery detection in a single language, but can also be extended to multi-language and multi-dialect audio forgery detection tasks, especially for local forgery in complex scenarios, and has broad application prospects;

[0024] 3. The present invention can be widely used in the fields of judicial identification, media review, etc., helping to quickly locate and confirm forged audio clips, providing more reliable technical support for audio forgery evidence collection, and has high application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0026] Figure 1 It is a schematic diagram of the system composition of the present invention;

[0027] Figure 2 Schematic diagram of the composition of the regional forged audio detection module based on contrastive learning in the present invention.

[0028] In the figure: 1. Multi-dimensional feature extraction module;

[0029] 2. Feature fusion module;

[0030] 3. Clustering-based frame-level pseudo-label generation module;

[0031] 4. Regional coarse positioning module

[0032] 5. Regional forged audio detection module based on contrastive learning; 51. Audio segmentation module; 52. Regional audio enhancement module; 53. Positive and negative sample construction module; 54. Loss function design module integrating contrast loss; 55. Regional forged audio detection module;

[0033] 6. Regional relocation module;

[0034] 7. Post-processing module; DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention;

[0036] Reference Figure 1-2 A forged audio region detection method based on feature clustering and contrastive learning includes a multi-dimensional feature extraction module 1, a feature fusion module 2, a clustering-based frame-level pseudo-label generation module 3, a region coarse positioning module 4, a region forged audio detection module 5 based on contrastive learning, a region relocation module 6, and a post-processing module 7.

[0037] Furthermore, the multi-dimensional feature extraction module 1 is used to extract multi-dimensional features from the audio signal;

[0038] Furthermore, the feature fusion module 2 is connected to the multi-dimensional feature extraction module 1, and is used to fuse the extracted multi-dimensional features, map the high-dimensional features to the low-dimensional space through the dimensionality reduction algorithm, and reduce redundant information;

[0039] Furthermore, a clustering-based frame-level pseudo-label generation module 3 is connected to the feature fusion module 2 and is used to divide the audio frames into two categories using an unsupervised clustering algorithm according to the fused features to generate frame-level pseudo-labels;

[0040] Furthermore, the region coarse positioning module 4 is connected to the clustering-based frame-level pseudo-label generation module 3, and is used to preliminarily classify the audio frames based on the pseudo-labels generated by clustering, and roughly locate the audio regions of different categories;

[0041] Furthermore, the regional forged audio detection module 5 based on contrastive learning is connected with the multi-dimensional feature extraction module 1, the regional rough positioning module 4, and the regional repositioning module 6 to perform in-depth detection on regional audios of different categories and determine the authenticity of the audios in the divided regions;

[0042] The regional forged audio detection module 5 based on contrastive learning includes:

[0043] The audio segmentation module 51 is connected to the region rough positioning module 4 and the region repositioning module 6, and is used to segment the audio into a plurality of substrings according to the result of the region positioning;

[0044] The regional audio enhancement module 52 is connected to the audio segmentation module 51 and is used to enhance the segmented audio substring samples through a variety of audio transformations to improve the diversity of the samples;

[0045] A positive and negative sample construction module 53, connected to the regional audio enhancement module 52, is used to construct positive and negative sample pairs based on samples of different categories after audio enhancement;

[0046] A loss function design module 54 integrating contrast loss is used to design an effective loss function for the forged audio detection model to improve the classification accuracy of the model;

[0047] The regional forged audio detection module 55 is connected to the positive and negative sample construction module 53, the loss function design module 54 of the fusion contrast loss, and the regional relocation module 6, and is used to train a model for authenticity detection of regional samples of the same category, and then use the trained model to relocate the authenticity region.

[0048] Furthermore, the region relocation module 6 is connected to the region forged audio detection module 5 based on contrastive learning, and is used to re-detect the audio at the frame level based on the detection model optimized by contrastive learning, further refine and adjust the boundaries of the true and false regions, and improve the detection accuracy;

[0049] Furthermore, the post-processing module 7 is connected to the regional forged audio detection module 6 based on contrastive learning, and is used to smooth and standardize the output results of the detection model, and output accurate forged region positioning and audio authenticity detection results.

[0050] The above-mentioned forged audio region detection method based on feature clustering and contrastive learning first extracts multi-dimensional information such as time domain, frequency domain and deep embedding features from the audio data, and fuses these features to fully capture the different levels of characteristics of the audio.

[0051] Next, a clustering method is used to perform preliminary classification of the audio data at the frame level to generate pseudo labels. Based on the pseudo labels, different categories of audio are roughly located. Different regions are divided according to categories, and the audio in the region is used as a unit to train the audio authenticity detection model through a comparative learning algorithm. In addition, the detection model is used to relocate the audio authenticity area, and the detection accuracy of the model and the division of regional boundaries are continuously optimized.

[0052] Finally, a forged audio region detection model based on frame-level multi-dimensional feature fusion was constructed. Based on this model, the output results were post-processed to obtain accurate forged region positioning and audio authenticity detection results.

[0053] This method is not only suitable for detecting forged audio in a single language, but can also be extended to audio scenarios with multiple languages ​​and accents. It is particularly suitable for application scenarios such as forensic identification and media review that require precise audio analysis, and has broad practical value.

[0054] In one embodiment of the present invention, a forged audio region detection method based on feature clustering and contrastive learning is proposed. Figure 1As shown, this embodiment includes: a multi-dimensional feature extraction module 1, used to extract multi-dimensional features from audio signals; a feature fusion module 2, connected to the multi-dimensional feature extraction module 1, used to fuse the extracted multi-dimensional features, map the high-dimensional features to the low-dimensional space through the dimensionality reduction algorithm, and reduce redundant information; a clustering-based frame-level pseudo-label generation module 3, connected to the feature fusion module 2, used to divide the audio frames into two categories according to the fused features using an unsupervised clustering algorithm to generate frame-level pseudo-labels; a region coarse positioning module 4, connected to the clustering-based frame-level pseudo-label generation module 3, used to preliminarily classify the audio frames based on the pseudo-labels generated by clustering, and roughly locate the audio regions of different categories; based on The regional forged audio detection module 5 based on contrastive learning is connected to the multi-dimensional feature extraction module 1, the regional coarse positioning module 4, and the regional repositioning module 6, and is used to perform in-depth detection on regional audios of different categories to determine the authenticity of the audios in the divided areas; the regional repositioning module 6 is connected to the regional forged audio detection module 5 based on contrastive learning, and is used to re-detect the audio at the frame level based on the detection model optimized by contrastive learning, further refine and adjust the boundaries of the true and false areas, and improve the detection accuracy; the post-processing module 7 is connected to the regional forged audio detection module 6 based on contrastive learning, and is used to smooth and standardize the output results of the detection model, and output accurate forged area positioning and audio authenticity detection results.

[0055] In this embodiment, the multi-dimensional feature extraction module 1 is used to extract multi-dimensional features from the audio signal, including time-frequency domain features, deep embedding features, text features, rhythm features, etc.

[0056] Among them, the time-frequency features capture the subtle changes of the audio signal in time and frequency, and extract the linear prediction cepstral coefficients and long-time spectrum features from the time-frequency spectrum.

[0057] Deep embedding features are extracted through deep neural networks, which can capture deeper speech patterns and traces of forgery in audio. Two types of deep embedding feature extraction are performed: deep acoustic embedding is extracted from the publicly available speech recognition pre-trained model, and deep speaker embedding that can capture the identity of the speaker is extracted through the publicly available speaker recognition model.

[0058] Text feature extraction is based on the text obtained by speech recognition technology, and the pre-trained model RoBERTa is used to extract semantic information for analyzing the inconsistency of the forged audio content.

[0059] Prosodic features are used to capture the intonation, speaking speed, stress and other phonological information in the audio, including fundamental frequency, speaking speed, pause and duration distribution, and stress pattern. These multi-dimensional features from different perspectives constitute a more comprehensive analysis of audio signals and capture potential anomalies in forged audio.

[0060] The feature fusion module 2 is connected to the multi-dimensional feature extraction module 1, and is used to fuse the extracted multi-dimensional features, map the high-dimensional features to the low-dimensional space through the dimensionality reduction algorithm, and reduce redundant information. For the task of locating the forged area of ​​audio data, in order to better preserve the local structure of high-dimensional data, the t-distribution-random neighbor embedding (t-SNE) method is used to fuse the multi-dimensional features and generate the corresponding low-dimensional representation. Since t-SNE is sensitive to hyperparameters, some samples are first selected from the training set, and the parameters such as the amplification factor, perplexity, and learning rate are experimentally verified, the optimal parameters are selected, and then the multi-dimensional features of all audio data are mapped to a low dimension.

[0061] The clustering-based frame-level pseudo-label generation module 3 is connected to the feature fusion module 2 and is used to divide the audio frames into two categories using an unsupervised clustering algorithm based on the fused features to generate frame-level pseudo-labels. The number of cluster centers is set to 2, that is, the audio frames are divided into two categories, corresponding to the hypothetical categories of real audio and fake audio. According to the feature vector of each audio frame, the K-means algorithm is used to assign each audio frame to a cluster center with a closer distance, and the pseudo-label of each frame is marked as 0 or 1 according to its assigned class. Here, it is impossible to distinguish which cluster contains real or fake samples.

[0062] The regional coarse positioning module 4 is connected to the clustering-based frame-level pseudo-label generation module 3, and is used to preliminarily classify the audio frames based on the pseudo-labels generated by clustering, and roughly locate the audio regions of different categories. Due to the limited accuracy and consistency of the unsupervised clustering algorithm, the generated pseudo-labels tend to fluctuate between adjacent frames, resulting in discontinuity of the audio regions of the same category. Therefore, the regional coarse positioning module needs to further process these discontinuous pseudo-labels to ensure that the frame labels in each audio region have higher consistency. First, the pseudo-labels are smoothed using a sliding window method. The labels of adjacent frames are counted according to a time window of a certain length, and the majority class is taken as the pseudo-label of all frames in the window, thereby reducing the volatility of the frame labels.

[0063] label smooth (t)=majority(label(tn),...,label(t+n))

[0064] Among them, t represents the time frame and n is the window size.

[0065] After smoothing, the originally scattered frame-level pseudo-labels are transformed into more coherent audio intervals by treating adjacent frames belonging to the same category (real or fake) as a continuous audio region.

[0066] Due to the limitations of the clustering algorithm, sometimes the boundaries of the regions may be misaligned. For those audio clips that are too short, it may be caused by noise or local errors in the clustering. To this end, the boundaries of the audio regions are adjusted. First, the frame energy is detected near the boundary. If the energy changes dramatically, it may mean that a new audio region begins, so the boundary position is corrected. Then, the dynamic time warping method is used to compare the features in the region with the features of the adjacent regions, and the frames with the most obvious feature differences near the boundary are used as the boundary position, thereby further optimizing the boundary division. Then, a threshold is set, and short clips below the threshold are merged to ensure that the final positioning result is more reliable and coherent. Finally, different audio regions can be obtained according to the processed boundary position, and each audio region is assigned a unified pseudo-label 0 / 1.

[0067] The regional forged audio detection module 5 based on contrastive learning is connected to the multi-dimensional feature extraction module 1, the regional rough positioning module 4, and the regional repositioning module 6, and is used to perform in-depth detection on regional audios of different categories to determine the authenticity of the audio in the divided area. The regional forged audio detection module 5 based on contrastive learning includes: an audio segmentation module 51, which is used to segment the audio into multiple substrings according to the results of regional positioning; a regional audio enhancement module 52, which is used to enhance the segmented audio substring samples through a variety of audio transformations to improve the diversity of samples; a positive and negative sample construction module 53, which is used to construct positive and negative sample pairs based on different categories of samples after audio enhancement; a loss function design module 54 that integrates contrast loss, which is used to design an effective loss function for the forged audio detection model to improve the classification accuracy of the model; a regional forged audio detection module 55, which is used to train a model for authenticity detection of regional samples of the same category, and then use the trained model to reposition the authenticity area.

[0068] The audio segmentation module 51 is connected to the region rough positioning module 4 and the region repositioning module 6, and is used to segment the audio into multiple substrings according to the result of the region positioning. Specifically, the module divides the audio region and segments different regions in each audio into multiple continuous substrings, and the audio frame category (real or fake) in each substring remains consistent, ensuring that the audio frames in each substring are from the same category.

[0069] The regional audio enhancement module 52 is connected to the audio segmentation module 51, and is used to enhance the segmented audio substring samples through multiple audio transformations to improve the diversity of the samples. This module uses the following audio enhancement methods to enhance each audio substring: (1) Time-frequency masking: For the audio spectrum and waveform, randomly select certain frequency segments and time periods, and set their values ​​to zero or reduce them. (2) Time-frequency scaling: For the audio spectrum and waveform, randomly select certain frequency segments and time periods, and stretch or compress them. (3) Volume adjustment: Amplify and reduce the amplitude of the audio substring. (4) Audio cropping: Select a part of the audio time period for random cropping, and then stretch the cropped area to the original length. (5) Mixed enhancement mode: Randomly select several of the above four enhancement methods and use them on the same audio substring.

[0070] The positive and negative sample construction module 53 is connected to the regional audio enhancement module 52, and is used to construct positive and negative sample pairs based on samples of different categories after audio enhancement, and provide training data for the contrastive learning model. Constructing positive and negative sample pairs is the core step in contrastive learning. By allowing the model to learn the differences between samples of different categories, the ability to detect forged audio is improved. Since the forged voice detection task is a binary classification task, samples of the same category are selected as positive samples, and samples of different categories are selected as negative samples. Specifically, the positive sample pair is selected from audio substrings of the same category (real or forged), wherein the audio substring includes the original data and the data enhanced by the regional audio enhancement module 52; the negative sample pair is selected from audio substrings of different categories (real or forged), wherein the audio substring includes the original data and the data enhanced by the regional audio enhancement module 52. When constructing positive and negative sample pairs, first select two audio substrings of similar duration to maintain the effectiveness of contrastive learning. Then, in order to avoid the model learning irrelevant patterns, a certain degree of randomness is introduced. For each pair of positive or negative samples, different enhancement methods are randomly selected or some audio substrings are randomly selected for pairing to ensure the diversity of sample pairs and prevent the model from overfitting specific features. In addition, in the process of constructing negative sample pairs, a hard negative sample mining strategy is introduced, that is, samples from one category with similar features to another category are selected as negative samples. Hard negative samples are more difficult to distinguish and can significantly improve the learning ability of the model.

[0071] The loss function design module 54 integrating contrast loss is used to design an effective loss function for the forged audio detection model to improve the classification accuracy of the model. The loss function is composed of a combination of self-supervised contrast loss and cross entropy loss. Specifically, the design of the loss function can be expressed as:

[0072] L=L contrastive +λL cross-entropy

[0073] Among them, λ is a hyperparameter that controls the weight ratio of contrast loss and cross entropy loss. The value of λ is adjusted through experiments. cross-entropy is the cross entropy loss, which is used for the final audio authenticity classification task. contrastive It is a self-supervised contrast loss, which is mainly used to learn the similarities and differences between samples, bringing positive sample pairs of the same category closer together and pushing negative sample pairs of different categories farther apart, helping the model better distinguish between real audio and forged audio, expressed as:

[0074]

[0075] Where P represents the set of positive sample pairs, and N represents the set of all sample pairs. i and h j is the feature vector representation of sample i and sample j that constitutes a positive sample pair, h i and h k It is the feature vector representation of sample i and sample k that constitutes a negative sample pair. sim(h i ,h j ) represents the similarity between samples i and j, usually calculated using cosine similarity. τ is a temperature hyperparameter.

[0076] The regional forged audio detection module 55 is connected to the positive and negative sample construction module 53, the loss function design module 54 of the fusion contrast loss, and the regional relocation module 6, and is used to train a model for authenticity detection of regional samples of the same category, and then use the trained model to relocate the authenticity region. This module is an end-to-end forged audio detection system, including a deep embedding feature extraction layer, an attention layer, and a graph network layer, and finally outputs a true and false binary classification result. The deep embedding feature extraction layer extracts speech embedding features based on the self-supervised pre-training model wav2vec2.0 model; the attention layer optimizes the input embedding features, and can automatically learn which frames have higher abnormal feature weights during training, and enhance attention to these frames. The graph neural network is used to enhance the model's modeling capabilities for local areas of audio sequences. Using the constructed fusion contrast loss loss function for training, the model can perform binary classification on each regional audio and output the corresponding authenticity label.

[0077] The region relocation module 6 is connected to the region forged audio detection module 5 based on contrastive learning, and is used to re-detect the audio at the frame level based on the detection model optimized by contrastive learning, further refine and adjust the boundaries of the true and false regions, and improve the detection accuracy. After the initial pseudo-label is generated, the boundary between the forged region and the real region may be unclear or overly smooth. Therefore, these frames are reclassified, and the frames in the boundary area are refined in particular. First, all audio data are processed frame by frame, and the true and false labels of each frame are generated based on the forged audio detection model inference. Then, this frame-level true and false label is fused with the pseudo-label generated by the region coarse positioning module 4. The specific fusion method is: first, the consistency of the two label sets is checked. If the labels of the same frame in the two results are consistent, the label is directly used as the final result; if a frame is inconsistent in the two label sets, the frame is locally adjusted based on the smoothness assumption of the time series. If the continuous frames before and after the frame are of the same category, the label of the conflicting frame is adjusted to the consistent category of the adjacent frames through the smoothing strategy, thereby eliminating the boundary fuzziness problem. Finally, the fusion result will be used as the relocated region label to provide a more accurate true and false region boundary. Using this more accurate boundary, the regional forged audio detection model based on contrastive learning is further optimized5, and the detection accuracy and regional positioning ability of the model are improved through retraining.

[0078] The post-processing module 7 is connected to the regional forged audio detection module 6 based on contrastive learning, and is used to smooth and standardize the output results of the detection model, and output accurate forged area positioning and audio authenticity detection results. The output results of the forged audio detection model will inevitably lead to isolated erroneous frame labels (noise) in the detection label due to data noise or model uncertainty. The post-processing module first processes these isolated erroneous labels. If the label of a frame is different from the labels of most of the frames around it, the label of the frame is adjusted to be consistent with the surrounding frames. In order to reduce the noise interference caused by frequent switching, for adjacent small areas, if the labels of these areas should logically belong to the same category, these areas are merged. For frames close to the decision boundary of the model, the detection threshold is dynamically adjusted to avoid excessive classification errors in uncertain areas.

[0079] The present invention proposes a forged audio region detection method based on feature clustering and contrastive learning, which is written in Python language. In addition, the present invention can be applied to computer terminals, handheld mobile devices or other forms of mobile devices.

Claims

1. A forged audio region detection method based on feature clustering and contrastive learning, characterized in that: It includes a multi-dimensional feature extraction module (1), a feature fusion module (2), a clustering-based frame-level pseudo-label generation module (3), a region coarse positioning module (4), a region forged audio detection module based on contrastive learning (5), a region relocation module (6), and a post-processing module (7).

2. The method for detecting forged audio regions based on feature clustering and contrastive learning according to claim 1, characterized in that: The multi-dimensional feature extraction module (1) is used to extract multi-dimensional features from audio signals.

3. The method for detecting forged audio regions based on feature clustering and contrastive learning according to claim 2, characterized in that: The feature fusion module (2) is connected to the multi-dimensional feature extraction module (1) and is used to fuse the extracted multi-dimensional features and map the high-dimensional features to a low-dimensional space through a dimensionality reduction algorithm, thereby reducing redundant information.

4. The method for detecting forged audio regions based on feature clustering and contrastive learning according to claim 3, characterized in that: The clustering-based frame-level pseudo-label generation module (3) is connected to the feature fusion module (2) and is used to divide the audio frames into two categories based on the fused features using an unsupervised clustering algorithm to generate frame-level pseudo-labels.

5. The method for detecting forged audio regions based on feature clustering and contrastive learning according to claim 4, characterized in that: The region coarse positioning module (4) is connected to the clustering-based frame-level pseudo-label generation module (3) and is used to perform preliminary classification of audio frames based on pseudo-labels generated by clustering, and roughly locate audio regions of different categories.

6. The method for detecting forged audio regions based on feature clustering and contrastive learning according to claim 5, characterized in that: The regional forged audio detection module (5) based on contrastive learning is connected to the multi-dimensional feature extraction module (1), the regional rough positioning module (4), and the regional repositioning module (6), and is used to perform in-depth detection on regional audio of different categories to determine the authenticity of audio in the divided area; The regional forged audio detection module (5) based on contrastive learning comprises: An audio segmentation module (51), connected to the region rough positioning module (4) and the region repositioning module (6), is used to segment the audio into a plurality of substrings according to the result of the region positioning; A regional audio enhancement module (52), connected to the audio segmentation module (51), is used to enhance the segmented audio substring samples through a variety of audio transformations to improve the diversity of the samples; A positive and negative sample construction module (53), connected to the regional audio enhancement module (52), is used to construct positive and negative sample pairs based on samples of different categories after audio enhancement; A loss function design module (54) integrating contrast loss is used to design an effective loss function for a forged audio detection model to improve the classification accuracy of the model; The regional forged audio detection module (55) is connected to the positive and negative sample construction module (53), the loss function design module (54) of the fusion contrast loss, and the regional relocation module (6), and is used to train a model for authenticity detection of regional samples of the same category, and then use the trained model to relocate the authenticity region.

7. The method for detecting forged audio regions based on feature clustering and contrastive learning according to claim 6, characterized in that: The region relocation module (6) is connected to the region forged audio detection module (5) based on contrastive learning, and is used to re-detect the audio at the frame level based on the detection model optimized by contrastive learning, further refine and adjust the boundaries of the true and false regions, and improve the detection accuracy.

8. The method for detecting forged audio regions based on feature clustering and contrastive learning according to claim 7, characterized in that: The post-processing module (7) is connected to the regional forged audio detection module (5) based on contrastive learning, and is used to perform smoothing and standardization processing on the output results of the detection model, and output accurate forged region positioning and audio authenticity detection results.

Images