Frame offset attention and multi-level classification-based segment-level synthetic speech detection method and system

Abstract

The invention discloses a fragment-level synthetic speech detection method and system based on frame offset attention and multi-level classification, and relates to the technical field of speech synthesis, and the method comprises the steps: obtaining speech features, and carrying out the preliminary processing; obtaining the similarity FOA features between the attention voice frames through a frame offset attention mechanism; extracting the FOA features to obtain multi-scale speech features; and obtaining a detection result of each segment in the multi-scale speech features and a detection result of the whole speech through the segment-level classification network and the utterance-level classification network. According to the method, cosine similarity values between each voice frame feature and the voice frame and between the voice frame feature and left and right neighbors are obtained by using cosine similarity, so that the similarity between feature vectors is measured. The fragment-level classification network focuses on fragment information; the utterance-level classification network aims to judge whether the whole voice contains the synthetic segments or not, and more attention is paid to the globality. And the model comprehensively considers the fragment information and the whole sentence information to obtain a more accurate classification result.

Description

Technical Field

[0001] The present invention relates to the field of speech synthesis technology, and in particular to a segment-level synthesized speech detection method and system based on frame offset attention and multi-level classification. Background Art

[0002] Automatic Speaker Verification (ASV) has been widely used in identity recognition tasks such as banks and smart homes. Criminals will use speech synthesis, speech conversion, and replay techniques to obtain synthetic speech to attack the ASV system. Therefore, researchers will defend against these attacks through synthetic speech detection systems. However, most of the current research on synthetic speech detection is aimed at the complete synthesized audio signal, while ignoring the situation where the synthesized or converted speech fragments are embedded in the real speech. This situation is called "partial deceptive speech" attack or "fragment-level synthetic speech" attack. The attacker can change the time, place, and other single words or words in a sentence to achieve the purpose of changing the semantics. At the same time, because the fragment-level synthetic speech contains a large number of real speech fragments, the fragment-level synthetic speech is more difficult to detect. Therefore, it is very important to design a model for detecting fragment-level synthetic speech.

[0003] Existing segment-level synthetic speech detection is mainly completed through the existing utterance-level synthetic speech detection model. However, the traditional utterance-level synthetic speech detection method is used for segment-level synthetic speech detection, that is, all segment features are merged to obtain a utterance feature and then classified. However, segment-level synthetic speech contains a large number of real speech segments. During training, back propagation will process all segment features according to the utterance-level labels, causing the real speech segments to be forced to move in the direction of the synthetic speech segments. In addition, the existing methods specifically used for segment-level synthetic speech detection often lack attention to the similarity between speech frames, and the segment-level detection results and utterance-level detection results are not fully utilized. Therefore, it is an urgent problem to design a segment-level synthetic speech detection model that can consider the similarity between speech frames and comprehensively utilize the utterance-level and segment-level classification results.

[0004] Existing technologies usually improve synthetic speech detection models from the following two aspects:

[0005] 1. Use the traditional speech-level synthetic speech detection model in segment-level synthetic speech detection. For example, Lv et al. [1] By fine-tuning the unsupervised pre-trained model Wav2Vec2.0 as the front end, and using the existing speech-level synthetic speech detection model ECAPA-TDNN [2] Used as a backend classification model for segment-level synthetic speech detection.

[0006] 2. Perform segment-level synthetic speech detection by finding the boundary between synthetic speech and real speech. For example, the DKU-DUKEECE system [3] The scoring strategy integrates three components: boundary detection model, frame-level anti-spoofing detection model and VAE model. The boundary detection model is used to identify connection boundaries in spliced ​​audio, the frame-level anti-spoofing detection model is used to determine the authenticity of a single audio frame, and the VAE model is used to identify outliers that deviate significantly from real data.

[0007] [1]Lv Z, Zhang S, Tang K, et al. Fake audio detection based on unsupervised pretraining models [C]. Proceeding of the IEEE International Conference on Acoustics, Speech and Signal Processing, Singapore, 2022:9231-9235.

[0008] [2] Desplanques B, Thienpondt J, Demuynck K. Ecapa-tdnn: emphasized channel attention, propagation and aggregation in tdnn based speakerverification[C]. Proceeding of the Interspeech, Shanghai, China, 2020: 3830-3834.

[0009] [3]Cai Z,Wang W,Wang Y,et al.The DKU-DUKEECE system for themanipulation region location task of ADD 2023[J].arXiv preprint arXiv:2308.10281,2023.

[0010] The above existing technologies have the following disadvantages: when the traditional speech-level synthetic speech detection model is used in segment-level synthetic speech detection, all segments in the speech are often merged to obtain a speech-level feature for classification. However, when training segment-level synthetic speech, the feature is processed by the speech-level label during the back propagation process, which causes the real speech segment to move in the direction of forgery. Although the method of segment-level synthetic speech detection by finding the boundary between the synthetic speech and the real speech takes the splicing into account, it lacks attention to the similarity between speech frames, and does not fully utilize the segment-level detection results and the speech-level detection results. Summary of the invention

[0011] The purpose of the present invention is to propose a segment-level synthetic speech detection method and system based on frame offset attention and multi-level classification, which focuses on the similarity between feature vectors and comprehensively considers segment information and whole sentence information.

[0012] According to a first aspect of an embodiment of the present disclosure, a segment-level synthetic speech detection method based on frame offset attention and multi-level classification is provided, comprising the following steps:

[0013] Acquire speech features and perform preliminary processing;

[0014] The FOA feature focusing on the similarity between speech frames is obtained through the frame offset attention mechanism;

[0015] Extract FOA features to obtain multi-scale speech features;

[0016] Get the detection results of each segment in the multi-scale speech feature and the detection results of the entire speech.

[0017] In one of the embodiments, speech features are obtained and preliminarily processed, specifically: the pre-trained model Wav2Vec2.0 is used to extract speech features, and the speech features are preliminarily processed through an embedding module.

[0018] In one embodiment, the FOA feature focusing on the similarity between speech frames is obtained through the frame offset attention mechanism, specifically:

[0019] Define the feature vector of each speech frame as E, where E∈R D×T , where D and T represent the dimension and number of frames in the initial speech feature phase respectively. The similarity S between adjacent speech feature vectors is obtained by formula (1):

[0020]

[0021] Where cos_sim() represents the cosine similarity between the two, and the cosine similarity between the current speech feature vector and its left neighbor, itself, and right neighbor is obtained in turn;

[0022] Get the local attention weight att i ,att i As shown in formula (2):

[0023] att i =[cos_sim(e i ,e i-1 ),cos_sim(e i ,e i ),cos_sim(e i ,e i+1 )] (2)

[0024] Where [...] means concatenating three cosine similarity values ​​into one vector; e i represents the current speech feature vector, e i-1 represents the left neighbor speech feature vector, e i+1 represents the right neighbor speech feature vector;

[0025] Concatenate all local attention weights att i , obtain the attention weight matrix A.

[0026] In one embodiment, when concatenating the local attention weights att i In the process, first the local attention weight att i The intermediate matrix Att is obtained by concatenation along the channel dimension, and then an all-zero matrix A with the same shape as the original feature X of the entire speech is created. Finally, the intermediate matrix Att is placed in the corresponding positions in the all-zero matrix A to obtain the attention weight matrix A;

[0027] After obtaining the attention weight matrix A, it is weighted with the original feature vector X of the speech through formula (3) to obtain the new feature vector X weight :

[0028] X weight =X⊙A(3)

[0029] Among them, ⊙ represents element-by-element multiplication; then the weighted features are operated as in formula (4) to obtain Feature FOA feature:

[0030] Feature FOA =(X weight ) T W T (4)

[0031] Where W∈R B,T,out_channels×KRepresents the weight in the convolution operation, B represents the batch size, T represents the number of feature frames, out_channels represents the number of output channels, and K represents the convolution kernel size.

[0032] In one embodiment, the FOA feature is extracted to obtain multi-scale speech features, specifically:

[0033] The FOA features are extracted by stacking N parallel convolution modules. The maximum pooling layer is introduced in the first N-1 parallel convolution modules to further extract features, and the global maximum pooling layer is used in the last parallel convolution module to obtain global features. Each parallel convolution module has a convolution layer with different expansion factors, corresponding batch normalization and ReLU activation function.

[0034] In one of the embodiments, the detection results of each segment in the multi-scale speech feature and the detection results of the entire speech are obtained through a segment-level classification network and an utterance-level classification network. The segment-level classification network predicts each segment in the multi-scale speech feature to distinguish between real speech segments and synthetic speech segments; the utterance-level classification network determines whether the entire multi-scale speech feature contains a synthetic segment.

[0035] In one of the embodiments, the segment-level classification network obtains a segment-level vector for speech prediction through an Edge module and an Edge_linear module, and uses an interpolation method to upsample the segment-level vector to a uniform size to obtain a final segment-level prediction result.

[0036] The speech-level classification network first uses a flattening layer to flatten the input speech features, and then uses three linear layers as fully connected layers to extract speech features and classification to obtain the final prediction probability.

[0037] According to a second aspect of an embodiment of the present disclosure, a segment-level synthetic speech detection system based on frame offset attention and multi-level classification is provided, including:

[0038] A preliminary processing module, which obtains speech features and performs preliminary processing;

[0039] The frame offset attention module obtains the FOA feature that focuses on the similarity between speech frames through the frame offset attention mechanism;

[0040] A multi-scale feature extraction module extracts FOA features to obtain multi-scale speech features;

[0041] The multi-level classification module obtains the detection results of each segment in the multi-scale speech features and the detection results of the entire speech.

[0042] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored and running on the memory, wherein when the processor executes the program, the method for segment-level synthetic speech detection based on frame offset attention and multi-level classification is implemented.

[0043] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method for detecting segment-level synthetic speech based on frame offset attention and multi-level classification is implemented.

[0044] Compared with the prior art, the above technical solution adopted by the present invention has the following advantages: the present invention proposes a segment-level synthetic speech detection method based on frame offset attention (FOA) and multi-level classification technology. The method uses the frame offset attention mechanism to capture the similarity information between speech frame features, and further extracts the FOA features in parallel at multiple scales, thereby obtaining richer and more comprehensive information. At the same time, the method also adopts a multi-level classification strategy, which takes into account both the detection results at the speech level and the detection results at the segment level, thereby effectively improving the performance of segment-level synthetic speech detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings in the specification, which constitute a part of the present application, are used to provide further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0046] Figure 1 The schematic diagram of the method for detecting segment-level synthetic speech based on frame offset attention and multi-level classification is shown;

[0047] Figure 2 This is the schematic diagram of the cosine similarity calculation process;

[0048] Figure 3 This is the structural principle diagram of the multi-scale feature extraction module. DETAILED DESCRIPTION

[0049] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0050] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0051] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0052] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and systems according to various embodiments of the present disclosure. It should be noted that each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of a code may include one or more executable instructions for implementing the logical functions specified in each embodiment. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the flowchart and / or block diagram, and the combination of boxes in the flowchart and / or block diagram can be implemented using a dedicated hardware-based system that performs a specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0053] Embodiment 1:

[0054] In real life scenarios, in addition to utterance-level synthesized speech in which the entire sentence is synthesized speech, there is also segment-level synthesized speech in which synthesized speech segments are inserted into real speech, which poses a greater threat to the speaker recognition system. This embodiment provides a segment-level synthesized speech detection method based on frame offset attention and multi-level classification to cope with this challenge; the method comprises the following steps:

[0055] S1. Acquire speech features and perform preliminary processing;

[0056] Specifically, the pre-trained model Wav2Vec2.0 is used to extract speech features. Wav2Vec2.0 is a speech representation learning method proposed by Facebook AI Research. It uses self-supervised learning to model unlabeled audio data, and then extracts speech features. The speech features are preliminarily processed through an embedding module to better capture feature information and obtain corresponding feature vectors. Its embedding module includes two convolutional layers.

[0057] S2. The FOA feature focusing on the similarity between speech frames is obtained through the frame offset attention mechanism;

[0058] Taking into account that there may be large differences in the similarity of speech frames at the adjacent boundaries of synthetic speech segments and real speech segments, and that there may also be differences in the similarity characteristics of speech frames within synthetic speech segments and real speech segments, in order to effectively utilize and capture the similarity differences between speech frames, the present invention uses frame offset attention to operate on the speech feature vectors obtained after preliminary processing. Specifically, cosine similarity is used to obtain the cosine similarity value between each speech frame feature and itself, its left neighbor, and its right neighbor, thereby measuring the similarity between feature vectors. This method can not only better capture the relationship between speech frame features, but also effectively represent their changes and consistency between different speech frames, providing a richer information basis for subsequent feature extraction. An example of cosine similarity calculation is shown in Figure 2 shown.

[0059] By using frame offset attention at the speech frame level, calculating the cosine similarity of the vector, and fully considering the relationship between the current position and the left and right neighbors, the obtained FOA feature has richer time frame information.

[0060] S3. Extract FOA features to obtain multi-scale speech features, focus on more comprehensive information, and obtain the final multi-scale features;

[0061] like Figure 3 As shown in the figure, in order to extract FOA features at multiple scales, N groups of convolutional layers with different expansion factors and corresponding batch normalization and ReLU activation functions are parallelized. In this way, information of different scales can be captured at the same time and the expression ability can be enhanced. Finally, the results of these N different convolutions are spliced ​​together along the channel dimension to achieve the purpose of integrating information of different scales so that the network can learn information at multiple scales at the same time, effectively improving the feature extraction ability of the network.

[0062] The FOA features are extracted by stacking N (for example, 4) parallel convolution modules to achieve full feature extraction and analysis. The maximum pooling layer is introduced in the first N-1 (for example, 3) parallel convolution modules to further extract features, and the global maximum pooling layer is used in the last parallel convolution module to obtain global features, providing richer features. Finally, the obtained multi-scale features are sent to the multi-level classification for classification operations.

[0063] S4. Obtain the detection result of each segment in the multi-scale speech feature and the detection result of the entire speech.

[0064] Specifically, it is achieved through a segment-level classification network and an utterance-level classification network; the segment-level classification network aims to predict each small segment in the multi-scale speech features to distinguish between real speech segments and synthetic speech segments. Therefore, the segment-level classification network focuses more on segment information; while the utterance-level classification network aims to determine whether the entire speech contains synthetic segments, and pays more attention to the overall situation. The combination of these two classification networks can classify and predict speech data at the segment level and utterance level respectively, so that the model comprehensively considers segment information and whole sentence information to obtain more accurate classification results.

[0065] The structure of the segment-level classification network is shown in Table 1. The purpose of the segment-level classification network is to obtain a prediction for each segment in the multi-scale speech feature. Therefore, this prediction is based on each segment. In order to better maintain the order of the speech feature segments, the pooled multi-scale speech features are retained as the input features of the network. Through the Edge module and the Edge_linear module, the predicted segment-level vector output can be obtained. In order to make the prediction result more accurate, the interpolation method is used to upsample the result to a uniform size to obtain the final segment-level prediction result. This output can predict whether each segment in the speech is real speech or synthesized speech, so it can be used to obtain segment-level error rates.

[0066] Table 1. Network structure of segment-level classification

[0067]

[0068] The structure of the utterance-level classification network is shown in Table 2. The purpose of the utterance-level classification network is to obtain a prediction that the speech contains a synthetic segment. Therefore, in order to better capture global information and related features, the input is first flattened using a flattening layer. Next, three linear layers are used as fully connected layers to extract speech features and classification. The first two fully connected layers extract more abstract and advanced features to improve the expressiveness and generalization ability of the model, and the last fully connected layer is used to obtain the final prediction probability.

[0069] Table 2 Discourse level classification network structure

[0070]

[0071] Embodiment 2:

[0072] This embodiment provides a segment-level synthetic speech detection system based on frame offset attention and multi-level classification, including:

[0073] A preliminary processing module, which obtains speech features and performs preliminary processing;

[0074] The frame offset attention module obtains the FOA feature that focuses on the similarity between speech frames through the frame offset attention mechanism;

[0075] A multi-scale feature extraction module extracts FOA features to obtain multi-scale speech features;

[0076] The multi-level classification module obtains the detection results of each segment in the multi-scale speech features and the detection results of the entire speech.

[0077] Embodiment three:

[0078] An electronic device includes a memory, a processor, and a computer program stored and running on the memory, wherein the processor implements the above-mentioned segment-level synthetic speech detection method based on frame offset attention and multi-level classification when executing the program, including:

[0079] Acquire speech features and perform preliminary processing;

[0080] The FOA feature focusing on the similarity between speech frames is obtained through the frame offset attention mechanism;

[0081] Extract FOA features to obtain multi-scale speech features;

[0082] Get the detection results of each segment in the multi-scale speech feature and the detection results of the entire speech.

[0083] Embodiment 4:

[0084] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned segment-level synthetic speech detection method based on frame offset attention and multi-level classification, comprising:

[0085] Acquire speech features and perform preliminary processing;

[0086] The FOA feature focusing on the similarity between speech frames is obtained through the frame offset attention mechanism;

[0087] Extract FOA features to obtain multi-scale speech features;

[0088] Get the detection results of each segment in the multi-scale speech feature and the detection results of the entire speech.

[0089] The present invention uses the PartialSpoof dataset to conduct segment-level synthetic speech detection experiments. The dataset is constructed based on the LA dataset of ASVspoof2019 and includes a training set, a validation set, and a test set. The training set contains 2580 real voices with a total of 250281 segments, and 22800 false voices with a total of 308126 segments; the validation set contains 2548 real voices with a total of 255261 segments, and 22296 false voices with a total of 281690 segments; the test set contains 7355 real voices with a total of 772832 segments, and 63882 false voices with a total of 718798 segments. In order to evaluate the performance of the deception detection system, two indicators are used: the default minimum concatenated detection cost function (min t-DCF) and the equal error rate (EER). EER refers to the situation where the proportion of real speech being incorrectly rejected and the proportion of synthetic speech being incorrectly accepted are equal. The smaller the EER value, the better the performance of the detection system. Min t-DCF is used to evaluate the joint influence of the synthetic speech detection system and the ASV system. The value ranges from 0 to 1, where 0 means that both the ASV system and the synthetic speech detection system have no errors, and 1 means that the synthetic speech detection system cannot improve the ASV system.

[0090] In the present invention, a pre-trained Hugging Face version of Wav2Vec2.0-XLS-R-300M is used, and the front-end weights are frozen. The front-end model is pre-trained on 436k hours of unlabeled real speech data of 128 voices.

[0091] The present invention uses the maximum length zero padding technology to solve the problem of different lengths of input audio. The longest speech duration in the PartialSpoof dataset is 21.03 seconds. The Wav2Vec2.0 feature dimension corresponding to the speech is (1050, 1024), and the number of 160ms segment-level labels is 132. During the training process, the Adam optimizer is used, β1 = 0.9, β2 = 0.999, and the weight decay is set to 10 -4 In this paper, the batch size is set to 64. After each iteration, the model is evaluated using the validation set and the model with the best result is retained. The early stopping function is set so that if there is no improvement on the validation set for 20 consecutive iterations, the training stops.

[0092] In order to demonstrate the effectiveness of the segment-level synthetic speech detection designed by the present invention, the present invention is compared with the current mainstream models at the utterance level and segment level. The results at the utterance level are shown in Table 3, and the results at the segment level are shown in Table 4. As can be seen from the table, the present invention performs well on the PartialSpoof dataset, which illustrates the effectiveness of the present invention in detecting segment-level synthetic speech attacks. From the final results, the effectiveness of the present invention is also proved.

[0093] Table 3 compares the performance of the current mainstream segment-level synthetic speech detection model at the utterance level.

[0094]

[0095] Table 4 compares the performance of the segment-level synthetic speech detection model with the current mainstream segment-level synthetic speech detection model.

[0096]

[0097] Those skilled in the art should understand that the modules or steps of the present disclosure can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present disclosure is not limited to any specific combination of hardware and software.

[0098] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0099] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Technical personnel in the relevant field should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.

Claims

1. A segment-level synthetic speech detection method based on frame offset attention and multi-level classification, characterized in that: The following steps are involved: Acquire speech features and perform preliminary processing; The FOA feature focusing on the similarity between speech frames is obtained through the frame offset attention mechanism; Extract FOA features to obtain multi-scale speech features; Get the detection results of each segment in the multi-scale speech feature and the detection results of the entire speech.

2. According to claim 1, the segment-level synthetic speech detection method based on frame offset attention and multi-level classification is characterized in that: The speech features are obtained and preliminarily processed, specifically: the pre-trained model Wav2Vec2.0 is used to extract the speech features, and the speech features are preliminarily processed through the embedding module.

3. According to claim 1, the segment-level synthetic speech detection method based on frame offset attention and multi-level classification is characterized in that: The FOA feature focusing on the similarity between speech frames is obtained through the frame offset attention mechanism, specifically: Define the feature vector of each speech frame as E, where E∈R D×T , where D and T represent the dimension and number of frames in the initial speech feature phase respectively. The similarity S between adjacent speech feature vectors is obtained by formula (1): Where cos_sim() represents the cosine similarity between the two, and the cosine similarity between the current speech feature vector and its left neighbor, itself, and right neighbor is obtained in turn; Get the local attention weight att i ,att i As shown in formula (2): att i =[cos_sim(and i ,and i-1 ),cos_sim(and i ,and i ),cos_sim(and i ,and i+1 )](2) Where [...] means concatenating three cosine similarity values ​​into one vector; e i represents the current speech feature vector, e i-1 represents the left neighbor speech feature vector, e i+1 represents the right neighbor speech feature vector; Concatenate all local attention weights att i , obtain the attention weight matrix A.

4. According to claim 3, the segment-level synthetic speech detection method based on frame offset attention and multi-level classification is characterized in that: In splicing local attention weights att i In the process, first the local attention weight att i The intermediate matrix Att is obtained by concatenation along the channel dimension, and then an all-zero matrix A with the same shape as the original feature X of the entire speech is created. Finally, the intermediate matrix Att is placed in the corresponding positions in the all-zero matrix A to obtain the attention weight matrix A; After obtaining the attention weight matrix A, it is weighted with the original feature vector X of the speech through formula (3) to obtain the new feature vector X weight : X weight =X⊙A (3) Among them, ⊙ represents element-by-element multiplication; then the weighted features are operated as in formula (4) to obtain Feature FOA feature: Feature FOA =(X weight ) T W T (4) Where W∈R B,T,out_channels×K Represents the weight in the convolution operation, B represents the batch size, T represents the number of feature frames, out_channels represents the number of output channels, and K represents the convolution kernel size.

5. According to claim 1, the segment-level synthetic speech detection method based on frame offset attention and multi-level classification is characterized in that: The FOA features are extracted to obtain multi-scale speech features, specifically: The FOA features are extracted by stacking N parallel convolution modules. The maximum pooling layer is introduced in the first N-1 parallel convolution modules to further extract features, and the global maximum pooling layer is used in the last parallel convolution module to obtain global features. Each parallel convolution module has a convolution layer with different expansion factors, corresponding batch normalization and ReLU activation function.

6. According to claim 1, the segment-level synthetic speech detection method based on frame offset attention and multi-level classification is characterized in that: The detection results of each segment in the multi-scale speech feature and the detection results of the entire speech are obtained through a segment-level classification network and an utterance-level classification network. The segment-level classification network predicts each segment in the multi-scale speech feature to distinguish between real speech segments and synthetic speech segments; the utterance-level classification network determines whether the entire multi-scale speech feature contains a synthetic segment.

7. The segment-level synthetic speech detection method based on frame shift attention and multi-level classification according to claim 6 is characterized in that: The segment-level classification network obtains a segment-level vector for speech prediction through an Edge module and an Edge_linear module, and uses an interpolation method to upsample the segment-level vector to a uniform size to obtain a final segment-level prediction result; The speech-level classification network first uses a flattening layer to flatten the input speech features, and then uses three linear layers as fully connected layers to extract speech features and classification to obtain the final prediction probability.

8. A fragment-level synthetic speech detection system based on frame-shift attention and multi-level classification, characterized in that include: A preliminary processing module, which obtains speech features and performs preliminary processing; The frame offset attention module obtains the FOA feature that focuses on the similarity between speech frames through the frame offset attention mechanism; A multi-scale feature extraction module extracts FOA features to obtain multi-scale speech features; The multi-level classification module obtains the detection results of each segment in the multi-scale speech features and the detection results of the entire speech.

9. An electronic device comprising a memory, a processor and a computer program stored and running on the memory, wherein when the processor executes the program, the method for segment-level synthetic speech detection based on frame offset attention and multi-level classification as described in any one of claims 1-7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the segment-level synthetic speech detection method based on frame offset attention and multi-level classification as described in any one of claims 1-7.

Images