A speech anti-spoofing method, system and device based on feature overlap segmentation graph convolution network and a storage medium

Abstract

The application discloses a speech anti-fraud method, system and device based on a feature overlap segmentation graph convolution network and a storage medium, and belongs to the technical field of speech graph signal processing. The method comprises the following steps: obtaining original speech data; inputting the original speech data into a pre-trained speech anti-fraud network model to obtain speech embedding features; obtaining a speech authenticity probability based on the speech embedding features, comparing the speech authenticity probability with a set threshold, and determining whether the original speech is real speech. Through the feature overlap segmentation graph convolution network module in the speech anti-fraud network model, the application can effectively capture the detailed information of different regions in the original speech data features, can simultaneously extract local neighborhood features and global information, can enhance the richness of feature representation, can prevent the loss of important information, and can effectively improve the judgment accuracy of real speech and the robustness of the speech anti-fraud network model.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of speech signal processing, and particularly relates to a speech anti-spoofing method, system and device based on a feature overlapping segmentation graph convolutional network and a storage medium. BACKGROUND

[0002] In the design process of an automatic speaker verification (ASV) anti-spoofing system, the main challenge is how to maintain the reliability of the system when facing unpredictable spoofing attacks. Therefore, it is crucial to design an efficient speech anti-spoofing system and extract features that can effectively distinguish between real speech and spoofing speech.

[0003] However, the traditional speech anti-spoofing system has many defects in distinguishing between real speech and spoofing speech. On the one hand, the traditional speech anti-spoofing system usually directly processes the multi-channel feature map output by the residual network, failing to fully exploit the local information of the feature map, which affects the accuracy and robustness of detection. On the other hand, the traditional speech anti-spoofing system fails to effectively process the complex relationship between feature maps, i.e., ignores the interaction between features in each channel dimension, resulting in inaccurate learning of spoofing features in speech signals and obvious limitations in dealing with diversified and complex spoofing attacks. SUMMARY

[0004] The purpose of the present application is to overcome the deficiencies in the prior art and provide a speech anti-spoofing method, system, device and storage medium based on a feature overlapping segmentation graph convolutional network, which fully utilizes the local information and global information of speech signals to improve the accuracy and reliability of anti-spoofing.

[0005] The present application provides the following technical solutions:

[0006] In a first aspect, a speech anti-spoofing method based on a feature overlapping segmentation graph convolutional network is provided, comprising: obtaining original speech data;

[0007] inputting the original speech data into a pre-trained speech anti-spoofing network model to obtain speech embedding features;

[0008] obtaining a speech authenticity probability based on the speech embedding features, and comparing the speech authenticity probability with a set threshold to determine whether the original speech is real speech;

[0009] The speech anti-spoofing network model includes a data augmentation module, a feature extraction module, a feature overlapping segmentation graph convolutional network module, and an embedding feature generation module.

[0010] The data enhancement module is configured to perform data enhancement on the original speech data to obtain original speech enhanced data.

[0011] The feature extraction module is configured to extract multi-channel features from the original speech enhanced data to obtain a multi-channel feature map.

[0012] The feature overlapping segmentation graph convolution network module is configured to perform two-dimensional convolution on the multi-channel feature map to obtain a convolution output feature, perform overlapping segmentation on the convolution output feature to obtain a sub-feature map, and perform convolution on the sub-feature map and the convolution output feature after splicing to obtain an output feature.

[0013] The multi-channel feature map and the output feature are connected in residual to obtain integrated features; and the embedding feature generation module is configured to generate speech embedding features according to the integrated features.

[0014] As an optional technical solution of the present application, the data enhancement on the original speech data comprises introducing linear and nonlinear noise to perform noise processing on the original speech data.

[0015] As an optional technical solution of the present application, the feature extraction module comprises a self-supervised module and a residual network module; the self-supervised module comprises a convolutional neural network and a transformer architecture, and is configured to extract features from the original speech enhanced data to obtain first features; and the residual network module comprises a plurality of residual blocks, and is configured to perform multi-channel feature extraction on the first features to obtain a multi-channel feature map.

[0016] As an optional technical solution of the present application, the feature overlapping segmentation graph convolution network module comprises a two-dimensional convolution layer, a feature overlapping segmentation block, and a convolution block; the two-dimensional convolution layer is configured to adjust the dimension of the multi-channel feature map to obtain a convolution output feature; the feature overlapping segmentation block is configured to perform overlapping segmentation on the convolution output feature, splice the obtained sub-feature map with the convolution output feature to construct a graph structure, and perform graph convolution to obtain second features; and the convolution block is configured to perform convolution on the second features to obtain an output feature.

[0017] As an optional technical solution of the present application, the overlapping segmentation on the convolution output feature, the splicing of the obtained sub-feature map with the convolution output feature to construct a graph structure, and the graph convolution to obtain second features comprise:

[0018] The convolution output feature is a three-dimensional feature, represented as wherein, T, F, and C respectively represent the time dimension, the frequency dimension, and the channel dimension.

[0019] A mask matrix with the same size as the convolution output feature is applied on the convolution output feature, and a plurality of boundary overlapping reserved window mask matrices are applied on the convolution output feature in an element-by-element multiplication manner to perform boundary overlapping segmentation in the time dimension and the frequency dimension, wherein the elements of the mask matrix are 1 or 0, representing the reservation and neglect of the feature respectively;

[0020] The time dimension and the frequency dimension are combined as a feature dimension, and the sub-feature map and the convolution output feature are spliced along the feature dimension after dimension transformation to obtain a two-dimensional feature map , the two-dimensional feature map contains nodes, and the dimension number of the node feature is ;

[0021] The projection score of each node is calculated and represented as:

[0022] ;

[0023] wherein, the projection score of the nth node is, the nth node is, the projection vector is;

[0024] The first nodes with the largest projection score are selected for reservation, wherein, k represents the pooling rate, and a pooling feature map containing nodes is obtained, represented as:

[0025] ;

[0026] wherein, the feature matrix of the nodes is, the projection score vector of the nodes is, the element-level multiplication is,

[0027] Based on the , a graph structure is constructed and the channel dimension feature thereof is mapped to a graph node to obtain a feature map containing nodes, and the dimension number of the node feature is ; The feature map is input into a graph convolution network;

[0028] The graph convolution network contains two convolution layers, and the propagation mode between the layers is represented as:

[0029] ;

[0030] wherein, represents a feature matrix of the lth layer, represents a feature matrix of the (l+1)th layer, represents an adjacency matrix, , I represents a unit matrix, and respectively represent and a degree matrix of represents a weight matrix of the lth layer linear transformation, represents a nonlinear activation function;

[0031] After the graph convolution network, the output contains nodes, and the dimension number of the node feature is a second feature .

[0032] As an optional technical solution of the application, the embedding feature generation module comprises a time-frequency heterogeneous graph block and a maximum graph operation block.

[0033] The time-frequency heterogeneous graph block is used to construct a graph structure in the time domain dimension and the frequency domain dimension of the integrated feature respectively, to obtain a time domain graph structure and a frequency domain graph structure, and to integrate the node information of the time domain graph structure and the frequency domain graph structure to obtain a time-frequency heterogeneous graph.

[0034] The maximum graph operation block comprises two parallel branches; each branch processes the time-frequency heterogeneous graph through a graph attention network to output branch features, and performs maximum value operation and node processing on the two branch features to obtain speech embedding features.

[0035] As an optional technical solution of the application, the speech authenticity probability is obtained based on the speech embedding features, and the speech authenticity probability is compared with a set threshold to determine whether the original speech is real speech, comprising:

[0036] The speech embedding features are mapped through a linear layer to obtain a prediction score vector , and two components of the prediction score vector respectively represent real speech prediction score and spoof speech prediction score;

[0037] The real speech prediction score generates a speech authenticity probability after a softmax operation;

[0038] The speech authenticity probability is compared with a set threshold, if the speech authenticity probability is greater than or equal to the set threshold, it is determined as real speech; otherwise, it is determined as spoof speech.

[0039] In a second aspect, a speech anti-spoofing system based on a feature overlap segmentation graph convolutional network is provided, comprising: a data acquisition module configured to acquire original speech data;

[0040] an embedded feature extraction module configured to input the original speech data into a pre-trained speech anti-spoofing network model to obtain speech embedded features;

[0041] a judgment module configured to obtain a speech authenticity probability based on the speech embedded features, compare the speech authenticity probability with a set threshold, and determine whether the original speech is real speech;

[0042] The speech anti-spoofing network model comprises a data enhancement module, a feature extraction module, a feature overlap segmentation graph convolutional network module, and an embedded feature generation module.

[0043] The data enhancement module is configured to perform a data enhancement operation on the original speech data to obtain original speech enhanced data.

[0044] The feature extraction module is configured to extract multi-channel features in the original speech enhanced data to obtain a multi-channel feature map.

[0045] The feature overlap segmentation graph convolutional network module is configured to perform a two-dimensional convolution operation on the multi-channel feature map to obtain a convolution output feature, perform overlap segmentation on the convolution output feature to obtain a sub-feature map, and perform a convolution operation on the sub-feature map and the convolution output feature after splicing to obtain an output feature.

[0046] The multi-channel feature map and the output feature are connected in residual to obtain integrated features; and the embedded feature generation module is configured to generate speech embedded features according to the integrated features.

[0047] In a third aspect, a speech anti-spoofing device based on a feature overlap segmentation graph convolutional network is provided, comprising a processor and a storage medium; the storage medium is configured to store instructions; and the processor is configured to operate according to the instructions to perform the steps of the speech anti-spoofing method based on the feature overlap segmentation graph convolutional network of the first aspect.

[0048] In a fourth aspect, a computer readable storage medium having a computer program stored thereon is provided, the program being executed by a processor to implement the steps of the speech anti-spoofing method based on the feature overlap segmentation graph convolutional network of the first aspect.

[0049] Compared with the prior art, the present application has the following beneficial effects:

[0050] The present invention provides a speech anti-spoofing method based on a feature overlapping segmentation graph convolutional network. The convolution output features are overlapped and segmented through a feature overlapping segmentation graph convolutional network module, and the segmented features are spliced ​​with the convolution output features before performing a graph convolution operation. This effectively helps the model identify key areas and local changes in the original speech data, ensures that the output features make full use of local information and global information, and improves the overall expressiveness of the output features; at the same time, it effectively captures the correlation between multi-channel feature maps and improves the model's ability to learn complex data; residual connection of the multi-channel feature maps and the output features can effectively prevent the loss of important information, so that the model exhibits stronger robustness when processing complex data. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 2 is a schematic diagram of the structure of the voice anti-spoofing network model in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0053] Example 1

[0054] This embodiment provides a voice anti-spoofing method based on a feature overlapping segmentation graph convolutional network. It specifically includes the following steps:

[0055] Step 1: Obtain original voice data.

[0056] Step 2: Input the original speech data into a pre-trained speech anti-spoofing network model to obtain speech embedding features.

[0057] like Figure 1 As shown, the speech anti-spoofing network model includes a data enhancement module, a feature extraction module, a feature overlapping segmentation graph convolutional network module and an embedded feature generation module.

[0058] (1) The data enhancement module is used to perform data enhancement operations on the original speech data to obtain original speech enhanced data. Specifically, the data enhancement module uses the pre-acquired RawBoost tool to introduce linear and nonlinear noise, additive noise related to pulse signals, and additive noise unrelated to stationary signals to perform noise processing on the original speech data.

[0059] (2) The feature extraction module is configured to extract multi-channel features in the original speech enhancement data to obtain a multi-channel feature map. Specifically, the feature extraction module comprises a self-supervised module and a residual network module. The self-supervised module comprises a convolutional neural network and a transformer architecture, and is configured to extract features in the original speech enhancement data to obtain first features. The residual network module comprises a plurality of residual blocks, and is configured to perform multi-channel feature extraction on the first features to obtain the multi-channel feature map.

[0060] Further, in the embodiment, the self-supervised module uses a wav2vec2.0 pre-training model provided by a Fairseq open source toolkit to extract the first features.

[0061] Further, in the embodiment, the residual network module comprises five residual blocks, and each residual block comprises a two-dimensional convolutional layer, a batch normalization layer, a SeLU activation function and a two-dimensional convolutional layer connected in sequence.

[0062] (3) The feature overlapping segmentation graph convolution network module is configured to perform two-dimensional convolution operation on the multi-channel feature map to obtain convolution output features, perform overlapping segmentation on the convolution output features to obtain sub-feature maps, perform convolution operation on the sub-feature maps and the convolution output features after splicing to obtain output features.

[0063] As shown in Figure 1 , the feature overlapping segmentation graph convolution network module comprises a two-dimensional convolutional layer, a feature overlapping segmentation block and a convolution block. The two-dimensional convolutional layer is configured to adjust the dimension of the multi-channel feature map and enhance the expression ability thereof to obtain the convolution output features. The feature overlapping segmentation block is configured to perform overlapping segmentation on the convolution output features, splice the obtained sub-feature maps and the convolution output features to construct a graph structure, and perform graph convolution operation to obtain second features. The convolution block is configured to perform convolution operation on the second features to obtain the output features.

[0064] In the embodiment, the convolution output features are segmented, the obtained sub-feature maps are spliced with the convolution output features to construct a graph structure, and the second features are obtained after graph convolution operation, as follows:

[0065] (3.1) The convolution output features are three-dimensional features, represented as , wherein, T, F and C represent the time dimension, the frequency dimension and the channel dimension, respectively.

[0066] A mask matrix of the same size as the convolution output features is applied to the convolution output features. Multiple overlapping window mask matrices are applied to the convolution output features by element-by-element multiplication to perform overlapping segmentation in both the time and frequency dimensions. This ensures that each sub-feature map retains local feature information while also including the overlap between adjacent regions. The elements of the mask matrix are either 1 or 0, indicating whether a feature is retained or ignored, respectively.

[0067] (3.2) Combine the time dimension and the frequency dimension as the feature dimension, transform the sub-feature map and the convolution output feature respectively, and then splice them along the feature dimension to obtain a two-dimensional feature map , two-dimensional feature map Include nodes, and the dimension of node features is .

[0068] (3.3) Perform a pooling operation on the spliced ​​two-dimensional feature map to achieve dimensionality reduction.

[0069] The graph pooling operation is performed on the two-dimensional feature map , build a graph structure model on the feature dimension, convert the feature data into a graph, and map the feature dimension features to graph nodes. The number of nodes is reduced through the graph pooling layer to reduce the computational complexity. The details are as follows:

[0070] Calculate the projection score of each node, expressed as:

[0071] ;

[0072] in, represents the projection score of the nth node, , represents the nth node, Represents the projection vector.

[0073] Select the front with the largest projection score nodes are reserved, among which, , k represents the pooling rate, which is used to control the number of retained nodes. The eigenvalues ​​of the nodes are weighted to obtain The pooled feature map of nodes is expressed as:

[0074] ;

[0075] in, express The feature matrix of each node, The projection score vector of nodes, represents element-wise multiplication, Represents the activation function.

[0076] (3.4) based on the , construct a graph structure and map its channel dimension features to graph nodes, to obtain a feature graph containing nodes, the dimension number of node features is , input the feature graph into a graph convolution network (GCN).

[0077] The graph convolution network contains two convolution layers, and the propagation mode between layers is represented as:

[0078] ;

[0079] wherein, represents the feature matrix of the lth layer, represents the feature matrix of the (l+1)th layer, represents the adjacency matrix, , I represents the unit matrix, and respectively represent the degree matrix of and , and represents the weight matrix of the linear transformation of the lth layer, represents a nonlinear activation function.

[0080] After passing through the graph convolution network, a second feature containing nodes is output, and the dimension number of node features is .

[0081] (4) The embedding feature generation module is used to generate speech embedding features according to integrated features.

[0082] Specifically, the multi-channel feature graph and the output feature are connected in residual, to obtain integrated features input into an embedding feature generation module. The embedding feature generation module includes a time-frequency heterogeneous graph block and a maximum graph operation block.

[0083] Further, the time-frequency heterogeneous graph block is used to construct a graph structure in the time domain dimension and the frequency domain dimension of the integrated feature respectively, to obtain a time domain graph structure and a frequency domain graph structure, and integrate the node information of the time domain graph structure and the frequency domain graph structure, to obtain a time-frequency heterogeneous graph.

[0084] The maximum graph operation block includes two parallel branches. Each branch processes the time-frequency heterogeneous graph through a graph attention network, outputs branch features, and performs maximum value operation and node processing on the two branch features to obtain speech embedding features.

[0085] ​Step 3: Obtain the speech authenticity probability based on the speech embedding feature, and compare the speech authenticity probability with the set threshold to determine whether the original speech is authentic. The details are as follows:

[0086] The speech embedding feature is mapped through a linear layer to obtain a prediction score vector , the two components of the prediction score vector represent the true speech prediction score and the deceptive speech prediction score respectively.

[0087] The true speech prediction score is subjected to a softmax operation to generate a speech authenticity probability.

[0088] The speech authenticity probability is compared with a set threshold. If the speech authenticity probability is greater than or equal to the set threshold, the speech is judged to be a real speech; otherwise, the speech is judged to be a deceptive speech.

[0089] Example 2

[0090] This example provides an experimental application of the method described in Example 1. The details are as follows:

[0091] After the original speech data is processed by the data enhancement module and feature extraction module, a multi-channel feature map is obtained. , the characteristic shape is .

[0092] Multi-channel feature map Input feature overlapping segmentation map convolutional network module, through the two-dimensional convolution layer multi-channel feature map Perform dimension adjustment while maintaining convolution output features and multi-channel feature maps The number of channels is the same, both are 64, and the characteristic shape of the convolution output feature is In this embodiment, the convolution kernel size of the two-dimensional convolution layer is , the step size is 1, no padding above and below, and the left and right padding is 1.

[0093] The convolution output feature input feature overlap segmentation block performs overlapping segmentation. By applying a mask matrix of the same size as the convolution output feature to the convolution output feature, the four boundary overlap retention window mask matrices are applied to the convolution output feature by element-by-element multiplication to achieve boundary overlap segmentation in the time-frequency dimension. In this embodiment, the number of overlapping elements is 4, the number of sub-feature maps is 4, and the feature shapes of the sub-feature maps are respectively 、 、 and .

[0094] After the sub-feature map and the convolution output features are transformed in dimension, they are concatenated along the feature dimension to obtain five two-dimensional feature components including the feature dimension and the channel dimension. The feature shapes are 、 、 、 and , the feature shape of the two-dimensional feature is .

[0095] The two-dimensional feature obtained by splicing is modeled as a graph structure, and the feature dimension is mapped to a graph node, and the obtained graph structure contains nodes, and the dimension number of the node feature is . The projection score of each node is calculated , and the top nodes with the highest score are selected for retention. Among them, , a graph structure containing nodes is obtained. And the feature values of the nodes are weighted by element-level multiplication, and finally a pooled feature map containing nodes and the dimension number of the node feature is .

[0096] The pooled feature map is modeled as a graph structure, and the channel dimension feature is mapped to a graph node, and the obtained graph structure contains nodes, and the dimension of the node feature is feature map , the feature map is input into the graph convolution network.

[0097] After passing through the graph convolution network, the output contains nodes, and the dimension number of the node feature is second feature .

[0098] The second feature is input into the convolution block, and is sequentially subjected to dimension transformation, batch normalization layer, SeLU activation function, and two-dimensional convolution layer with convolution kernel size , step length 1, no padding up and down, and left and right padding 1. The output feature obtained is , and the feature shape is .

[0099] The multi-channel feature map is connected with the output feature in residual connection to obtain integrated feature , and the feature shape is .

[0100] The integrated feature is input into the embedding feature generation module to obtain the speech embedding feature , wherein , represents the dimension of the embedding feature. Based on the speech embedding feature, a speech authenticity probability is obtained, and the speech authenticity probability is compared with a set threshold value 0.5 to determine whether the original speech is real speech.

[0101] In this embodiment, the Equal Error Rate (EER) and the minimum tandem Detection Cost Function (min t-DCF) are used as evaluation indexes of the judgment result. Based on the above two indexes, the experimental results of the method are compared with the experimental results of the speech anti-fraud judgment under the Wav2vec-AASIST system, as shown in Table 1:

[0102] Table 1 Comparison results of EER and min t-DCF

[0103] min t-DCF EER (%) Wav2vec-AASIST 0.2134 1.06 The present method 0.2054 0.76

[0104] From Table 1, it can be seen that the method improves the accuracy and reliability of speech anti-fraud judgment by 28.30% and 3.75% respectively in terms of EER and min t-DCF compared with the Wav2vec-AASIST system.

[0105] Embodiment 3

[0106] The embodiment provides a speech anti-fraud system based on a feature overlap segmentation graph convolutional network, comprising:

[0107] A data acquisition module is configured to acquire original speech data.

[0108] An embedding feature extraction module is configured to input the original speech data into a pre-trained speech anti-fraud network model to obtain speech embedding features.

[0109] A judgment module is configured to obtain a speech authenticity probability based on the speech embedding features, and compare the speech authenticity probability with a set threshold value to determine whether the original speech is real speech.

[0110] The speech anti-fraud network model comprises a data enhancement module, a feature extraction module, a feature overlap segmentation graph convolutional network module, and an embedding feature generation module.

[0111] The data enhancement module is configured to perform a data enhancement operation on the original speech data to obtain original speech enhanced data.

[0112] The feature extraction module is configured to extract multi-channel features in the original speech enhanced data to obtain a multi-channel feature map.

[0113] The feature overlap segmentation graph convolution network module is configured to perform a two-dimensional convolution operation on the multi-channel feature map to obtain a convolution output feature, perform overlap segmentation on the convolution output feature to obtain a sub-feature map, and perform a convolution operation on the sub-feature map and the convolution output feature after splicing to obtain an output feature.

[0114] The multi-channel feature map and the output feature are connected in residual to obtain integrated features; and the embedding feature generation module is configured to generate a speech embedding feature according to the integrated features.

[0115] Embodiment 4

[0116] The embodiment provides a speech anti-spoofing device based on a feature overlap segmentation graph convolution network, including a processor and a storage medium; the storage medium is configured to store instructions; and the processor is configured to operate according to the instructions to perform steps of the speech anti-spoofing method based on the feature overlap segmentation graph convolution network in the embodiment 1.

[0117] Embodiment 5

[0118] The embodiment provides a computer-readable storage medium, which stores a computer program, and the program is executed by a processor to implement steps of the speech anti-spoofing method based on the feature overlap segmentation graph convolution network in the embodiment 1.

[0119] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROMs, optical storage devices, etc.) containing computer-usable program code.

[0120] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device implemented in the flowcharts and / or block diagrams. Figure 1 The device for implementing the function specified in one flow or multiple flows and / or blocks Figure 1 The device for implementing the function specified in one flow or multiple flows and / or blocks

[0121] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flow Figure 1 The functions of a flow or multiple flows and / or a block or multiple blocks in accordance with the present application can be implemented with software, hardware or with a combination of software and / or firmware. Figure 1

[0122] The software can be stored on one or more computer readable media, however, the computer readable media presented herein are only exemplary. For example, computer readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired computer program code in the form of computer readable instructions. Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow Figure 1 The functions of a flow or multiple flows and / or a block or multiple blocks in accordance with the present application can be implemented with software, hardware or with a combination of software and / or firmware.

[0123] The above description is merely illustrative of the preferred embodiments of the present application and various modifications and changes can be suggested to one skilled in the art, all of which fall within the scope of the present application.​

Claims

1. A voice anti-spoofing method based on feature overlap segmentation graph convolutional network, characterized in that, The method comprises: obtaining original speech data; inputting the original speech data into a pre-trained speech anti-deception network model to obtain speech embedding features; obtaining a speech authenticity probability based on the speech embedding features, and comparing the speech authenticity probability with a set threshold to determine whether the original speech is real speech; wherein the speech anti-deception network model comprises a data enhancement module, a feature extraction module, a feature overlapping segmentation graph convolution network module, and an embedding feature generation module; the data enhancement module is configured to perform data enhancement operations on the original speech data to obtain original speech enhancement data; the feature extraction module is configured to extract multi-channel features from the original speech enhancement data to obtain a multi-channel feature map; the feature overlapping segmentation graph convolution network module is configured to perform two-dimensional convolution operations on the multi-channel feature map to obtain convolution output features, and perform overlapping segmentation on the convolution output features to obtain sub-feature maps, and perform convolution operations on the sub-feature maps and the convolution output features after splicing to obtain output features; the multi-channel feature map and the output features are connected in residual connection to obtain integrated features; and the embedding feature generation module is configured to generate speech embedding features according to the integrated features. 2.The voice anti-spoofing method based on feature overlap segmentation graph convolutional network according to claim 1, wherein: The data enhancement operations on the original speech data comprise introducing linear and nonlinear noise to perform noise processing on the original speech data. 3.The voice anti-spoofing method based on feature overlap segmentation graph convolutional network according to claim 1, characterized in that, The feature extraction module comprises a self-supervised module and a residual network module; the self-supervised module comprises a convolutional neural network and a transformer architecture, and is configured to extract features from the original speech enhancement data to obtain first features; the residual network module comprises a plurality of residual blocks, and is configured to perform multi-channel feature extraction on the first features to obtain a multi-channel feature map. 4.The voice anti-spoofing method based on feature overlap segmentation graph convolutional network according to claim 1, characterized in that, The feature overlapping segmentation graph convolution network module comprises a two-dimensional convolution layer, a feature overlapping segmentation block, and a convolution block; the two-dimensional convolution layer is configured to adjust the dimensions of the multi-channel feature map to obtain convolution output features; the feature overlapping segmentation block is configured to perform overlapping segmentation on the convolution output features, splice the obtained sub-feature maps with the convolution output features to construct a graph structure, and perform graph convolution operations to obtain second features; the convolution block is configured to perform convolution operations on the second features to obtain output features.

5. The voice anti-spoofing method based on feature overlap segmentation graph convolutional network according to claim 4, characterized in that, The overlapping segmentation on the convolution output features, the splicing of the obtained sub-feature maps with the convolution output features to construct a graph structure, and the graph convolution operations to obtain second features comprise: The convolution output feature is a three-dimensional feature, represented as wherein, respectively represent the time dimension, the frequency dimension, and the channel dimension. applying a mask matrix with the same size as the convolution output features to the convolution output features, and applying a plurality of boundary overlapping reserved window mask matrices to the convolution output features in an element-by-element multiplication manner to perform boundary overlapping segmentation in the time dimension and the frequency dimension, wherein the elements of the mask matrix are 1 or 0, representing the reservation and neglect of the features, respectively; The time dimension and the frequency dimension are combined as a feature dimension, and the sub-feature maps and the convolution output features are respectively subjected to dimension transformation and then spliced along the feature dimension to obtain a two-dimensional feature map , the two-dimensional feature map contains nodes, and the dimension number of the node features is ; calculating the projection score of each node, represented as: ； wherein, denotes the projection score of the n-th node, denotes the n-th node, denotes the projection vector; Select the front with the largest projection score nodes are reserved, among which, , k represents the pooling rate, and the obtained The pooled feature map of nodes is expressed as: ； wherein, represents a feature matrix of the nodes, a projection score vector of the nodes, represents an element-wise multiplication, represents an activation function; Based on the , the graph structure is constructed and its channel dimension features are mapped to graph nodes to obtain a feature graph containing nodes, the dimension number of node features being The feature graph is input into a graph convolution network.​ the graph convolution network comprises two convolution layers, and the propagation mode between the layers is represented as: ； wherein, denotes the feature matrix of the l-th layer, denotes the feature matrix of the l+1-th layer, denotes the adjacency matrix, I denotes the identity matrix, and denote the degree matrix of and respectively, denotes the weight matrix of the l-th layer linear transformation, denotes a non-linear activation function; After the graph convolution network, an output contains nodes, the dimension number of node features is second features .

6. The speech anti-spoofing method based on feature overlap segmentation graph convolutional network according to claim 1, characterized in that, the embedding feature generation module comprises a time-frequency heterogeneous graph block and a maximum graph operation block; The time-frequency heterogeneous graph is used for constructing a graph structure in a time domain dimension and a frequency domain dimension respectively, obtaining a time domain graph structure and a frequency domain graph structure, and integrating node information of the time domain graph structure and the frequency domain graph structure to obtain a time-frequency heterogeneous graph; The maximum graph operation block includes two parallel branches; each branch processes the time-frequency heterogeneous graph through a graph attention network to output branch features, and performs maximum value operation and node processing on the two branch features to obtain speech embedding features.

7. The speech anti-spoofing method based on feature overlap segmentation graph convolutional network according to claim 1, characterized in that, The speech authenticity probability is obtained based on the speech embedding features, and the speech authenticity probability is compared with a set threshold to determine whether the original speech is real speech, including: map the voice embedding features through a linear layer to obtain a prediction score vector two components of the prediction score vector represent a real voice prediction score and a spoof voice prediction score, respectively; The real speech prediction score is subjected to a softmax operation to generate a speech authenticity probability; The speech authenticity probability is compared with a set threshold, if the speech authenticity probability is greater than or equal to the set threshold, the original speech is determined to be real speech; otherwise, the original speech is determined to be a fake speech.

8. A voice anti-spoofing system based on feature overlap segmentation graph convolutional network, characterized in that, It includes: A data acquisition module is configured to acquire original speech data; An embedding feature extraction module is configured to input the original speech data into a pre-trained speech anti-cheating network model to obtain speech embedding features; A judgment module is configured to obtain a speech authenticity probability based on the speech embedding features, and compare the speech authenticity probability with a set threshold to determine whether the original speech is real speech; The speech anti-cheating network model includes a data enhancement module, a feature extraction module, a feature overlapping segmentation graph convolution network module, and an embedding feature generation module; The data enhancement module is configured to perform data enhancement operation on the original speech data to obtain original speech enhanced data; The feature extraction module is configured to extract multi-channel features in the original speech enhanced data to obtain a multi-channel feature map; The feature overlapping segmentation graph convolution network module is configured to perform two-dimensional convolution operation on the multi-channel feature map to obtain convolution output features, perform overlapping segmentation on the convolution output features to obtain sub-feature maps, and perform convolution operation on the sub-feature maps and the convolution output features after splicing to obtain output features; The multi-channel feature map and the output features are connected in residual to obtain integrated features; and the embedding feature generation module is configured to generate speech embedding features according to the integrated features. 9.A speech anti-spoofing device based on a feature overlap segmentation graph convolutional network, characterized in that, It includes a processor and a storage medium; The storage medium is configured to store instructions; The processor is configured to operate according to the instructions to perform the steps of the speech anti-cheating method based on the feature overlapping segmentation graph convolution network according to any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the speech anti-cheating method based on the feature overlapping segmentation graph convolution network according to any one of claims 1-7.

Images