Voice anti-spoofing method, system and device based on feature overlapping segmentation graph convolutional network, and storage medium

Abstract

The invention discloses a voice anti-cheating method, system and device based on a feature overlapping segmentation graph convolutional network, and a storage medium, and belongs to the technical field of voice graph signal processing. The method comprises the steps of obtaining original voice data; inputting the original voice data into a pre-trained voice anti-deception network model to obtain voice embedding features; and obtaining a voice authenticity probability based on the voice embedding feature, comparing the voice authenticity probability with a set threshold value, and judging whether the original voice is real voice or not. According to the method, detail information of different areas in original voice data features is effectively captured through a feature overlapping segmentation graph convolutional network module in a voice anti-deception network model, local neighborhood features and global information can be extracted at the same time, the richness of feature representation is enhanced, important information is prevented from being lost, and the robustness of voice anti-deception is improved. And the judgment accuracy of the real voice and the robustness of the voice anti-deception network model are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of speech graph signal processing, and specifically relates to a speech anti-spoofing method, system, device and storage medium based on a feature overlapping segmentation graph convolutional network. Background Art

[0002] In the design process of Automatic Speaker Verification (ASV) anti-spoofing system, the main challenge is how to maintain the reliability of the system in the face of unpredictable spoofing attacks. Therefore, it is crucial to design an efficient voice anti-spoofing system and extract features that can effectively distinguish between real voice and spoofed voice.

[0003] However, traditional voice anti-spoofing systems have many defects in distinguishing real speech from spoofed speech. On the one hand, traditional voice anti-spoofing systems usually directly process the multi-channel feature maps output by the residual network, failing to fully exploit the local information of the feature maps, affecting the accuracy and robustness of detection. On the other hand, traditional voice anti-spoofing systems fail to effectively handle the complex relationship between feature maps, that is, they ignore the interaction between the features of each channel dimension, resulting in inaccurate learning of the deception features in the speech signal, showing obvious limitations when dealing with diverse and complex deception attacks. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art and to provide a voice anti-spoofing method, system, device and storage medium based on a feature overlapping segmentation graph convolutional network, which fully utilizes the local and global information of the voice signal to improve the accuracy and reliability of anti-spoofing.

[0005] The present invention provides the following technical solutions:

[0006] In a first aspect, a voice anti-spoofing method based on a feature overlapping segmentation graph convolutional network is provided, comprising: obtaining original voice data;

[0007] Inputting the original speech data into a pre-trained speech anti-spoofing network model to obtain speech embedding features;

[0008] Obtaining a speech authenticity probability based on the speech embedding feature, and comparing the speech authenticity probability with a set threshold to determine whether the original speech is a true speech;

[0009] The speech anti-spoofing network model includes a data enhancement module, a feature extraction module, a feature overlapping segmentation graph convolutional network module and an embedded feature generation module;

[0010] The data enhancement module is used to perform data enhancement operation on the original speech data to obtain original speech enhanced data;

[0011] The feature extraction module is used to extract multi-channel features in the original speech enhancement data to obtain a multi-channel feature map;

[0012] The feature overlapping segmentation graph convolutional network module is used to perform a two-dimensional convolution operation on the multi-channel feature graph to obtain a convolution output feature, perform overlapping segmentation on the convolution output feature to obtain a sub-feature graph, and perform a convolution operation after splicing the sub-feature graph with the convolution output feature to obtain an output feature;

[0013] The multi-channel feature map and the output feature are residually connected to obtain an integrated feature; the embedded feature generation module is used to generate a speech embedded feature based on the integrated feature.

[0014] As an optional technical solution of the present invention, the data enhancement operation performed on the original speech data includes: introducing linear and nonlinear noise to perform noise processing on the original speech data.

[0015] As an optional technical solution of the present invention, the feature extraction module includes a self-supervision module and a residual network module; the self-supervision module includes a convolutional neural network and a transformer architecture, which are used to extract features from the original speech enhancement data to obtain a first feature; the residual network module includes multiple residual blocks, which are used to perform multi-channel feature extraction on the first feature to obtain a multi-channel feature map.

[0016] As an optional technical solution of the present invention, the feature overlapping segmentation graph convolutional network module includes a two-dimensional convolutional layer, a feature overlapping segmentation block and a convolution block; the two-dimensional convolutional layer is used to adjust the dimension of the multi-channel feature map to obtain the convolution output feature; the feature overlapping segmentation block is used to perform overlapping segmentation on the convolution output feature, concatenate the obtained sub-feature map with the convolution output feature to construct a graph structure, and obtain the second feature after performing a graph convolution operation; the convolution block is used to perform a convolution operation on the second feature to obtain the output feature.

[0017] As an optional technical solution of the present invention, overlapping segmentation is performed on the convolution output features, the obtained sub-feature graph is spliced ​​with the convolution output features to construct a graph structure, and the second feature is obtained after the graph convolution operation, including:

[0018] The convolution output feature is a three-dimensional feature, expressed as ,in, Represent the time dimension, frequency dimension and channel dimension respectively;

[0019] Applying a mask matrix of the same size as the convolution output feature to the convolution output feature, and applying multiple boundary overlap retention window mask matrices to the convolution output feature by element-by-element multiplication to perform boundary overlap segmentation in the time dimension and the frequency dimension, wherein the elements of the mask matrix are 1 or 0, respectively indicating retention and neglect of the feature;

[0020] The time dimension and the frequency dimension are combined as the feature dimension, and the sub-feature map and the convolution output feature are transformed in dimension and then concatenated along the feature dimension to obtain a two-dimensional feature map. , two-dimensional feature map Include nodes, and the dimension of node features is ;

[0021] Calculate the projection score of each node, expressed as:

[0022] ;

[0023] in, represents the projection score of the nth node, , represents the nth node, represents the projection vector;

[0024] Select the front with the largest projection score nodes are reserved, among which, , k represents the pooling rate, and the obtained The pooled feature map of nodes is expressed as:

[0025] ;

[0026] in, express The feature matrix of each node, The projection score vector of nodes, represents element-wise multiplication, represents the activation function;

[0027] Based on the , construct the graph structure and map its channel dimension features into graph nodes, and obtain nodes, and the number of dimensions of node features is Feature map , the feature map Input graph convolutional network;

[0028] The graph convolutional network contains two convolutional layers, and the propagation between layers is expressed as:

[0029] ;

[0030] in, represents the feature matrix of the lth layer, represents the feature matrix of the l+1th layer, represents the adjacency matrix, , I represents the identity matrix, and Respectively and The degree matrix of represents the weight matrix of the linear transformation of the lth layer, represents a nonlinear activation function;

[0031] After passing through the graph convolutional network, the output contains nodes, and the dimension of node features is The second characteristic .

[0032] As an optional technical solution of the present invention, the embedding feature generation module includes a time-frequency heterogeneous graph block and a maximum graph operation block;

[0033] The time-frequency heterogeneous graph block is used to construct graph structures in the time domain dimension and the frequency domain dimension of the integrated features respectively, to obtain a time domain graph structure and a frequency domain graph structure, and to integrate the node information of the time domain graph structure and the frequency domain graph structure to obtain a time-frequency heterogeneous graph;

[0034] The maximum graph operation block includes two parallel branches; each branch processes the time-frequency heterogeneous graph through a graph attention network, outputs branch features, performs maximum operation and node processing on the two branch features, and obtains speech embedding features.

[0035] As an optional technical solution of the present invention, the method of obtaining the speech authenticity probability based on the speech embedding feature and comparing the speech authenticity probability with a set threshold to determine whether the original speech is a real speech includes:

[0036] The speech embedding feature is mapped through a linear layer to obtain a prediction score vector , the two components of the prediction score vector represent the real speech prediction score and the deceptive speech prediction score respectively;

[0037] The real speech prediction score is subjected to a softmax operation to generate a speech authenticity probability;

[0038] The speech authenticity probability is compared with a set threshold. If the speech authenticity probability is greater than or equal to the set threshold, it is judged to be a real speech; otherwise, it is judged to be a deceptive speech.

[0039] In a second aspect, a voice anti-spoofing system based on a feature overlapping segmentation graph convolutional network is provided, comprising: a data acquisition module for acquiring original voice data;

[0040] An embedding feature extraction module, used for inputting the original voice data into a pre-trained voice anti-spoofing network model to obtain voice embedding features;

[0041] A judgment module, used to obtain a speech authenticity probability based on the speech embedding feature, and compare the speech authenticity probability with a set threshold to determine whether the original speech is a real speech;

[0042] The speech anti-spoofing network model includes a data enhancement module, a feature extraction module, a feature overlapping segmentation graph convolutional network module and an embedded feature generation module;

[0043] The data enhancement module is used to perform data enhancement operation on the original speech data to obtain original speech enhanced data;

[0044] The feature extraction module is used to extract multi-channel features in the original speech enhancement data to obtain a multi-channel feature map;

[0045] The feature overlapping segmentation graph convolutional network module is used to perform a two-dimensional convolution operation on the multi-channel feature graph to obtain a convolution output feature, perform overlapping segmentation on the convolution output feature to obtain a sub-feature graph, and perform a convolution operation after splicing the sub-feature graph with the convolution output feature to obtain an output feature;

[0046] The multi-channel feature map and the output feature are residually connected to obtain an integrated feature; the embedded feature generation module is used to generate a speech embedded feature based on the integrated feature.

[0047] In a third aspect, a speech anti-spoofing device based on a feature overlapping segmentation graph convolutional network is provided, comprising a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the speech anti-spoofing method based on a feature overlapping segmentation graph convolutional network described in the first aspect.

[0048] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of the speech anti-spoofing method based on the feature overlapping segmentation graph convolutional network described in the first aspect are implemented.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] The present invention provides a speech anti-spoofing method based on a feature overlapping segmentation graph convolutional network. The convolution output features are overlapped and segmented through a feature overlapping segmentation graph convolutional network module, and the segmented features are spliced ​​with the convolution output features to perform a graph convolution operation, which effectively helps the model identify key areas and local changes in the original speech data, ensures that the output features make full use of local information and global information, and improves the overall expression ability of the output features; at the same time, it effectively captures the correlation between multi-channel feature maps and improves the model's learning ability for complex data; residual connection of the multi-channel feature maps and the output features can effectively prevent the loss of important information, so that the model exhibits stronger robustness when processing complex data. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a structural diagram of a voice anti-spoofing network model in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0053] Example 1

[0054] This embodiment provides a voice anti-spoofing method based on a feature overlapping segmentation graph convolutional network. Specifically, it includes the following steps:

[0055] Step 1: Get the original voice data.

[0056] Step 2: Input the original speech data into a pre-trained speech anti-spoofing network model to obtain speech embedding features.

[0057] like Figure 1 As shown, the speech anti-spoofing network model includes a data enhancement module, a feature extraction module, a feature overlapping segmentation graph convolutional network module and an embedded feature generation module.

[0058] (1) The data enhancement module is used to perform data enhancement operations on the original speech data to obtain original speech enhanced data. Specifically, the data enhancement module introduces linear and nonlinear noise, additive noise related to pulse signals, and additive noise unrelated to stationary signals to perform noise processing on the original speech data through the pre-acquired RawBoost tool.

[0059] (2) The feature extraction module is used to extract multi-channel features from the original speech enhancement data to obtain a multi-channel feature map. Specifically, the feature extraction module includes a self-supervision module and a residual network module; the self-supervision module includes a convolutional neural network and a transformer architecture, which are used to extract features from the original speech enhancement data to obtain a first feature; the residual network module includes a plurality of residual blocks, which are used to perform multi-channel feature extraction on the first feature to obtain a multi-channel feature map.

[0060] Furthermore, in this embodiment, the self-supervision module uses the wav2vec2.0 pre-trained model provided by the Fairseq open source toolkit to extract the first feature.

[0061] Furthermore, in this embodiment, the residual network module includes 5 residual blocks, each residual block includes a two-dimensional convolutional layer, a batch normalization layer, a SeLU activation function and a two-dimensional convolutional layer connected in sequence.

[0062] (3) The feature overlapping segmentation map convolutional network module is used to perform a two-dimensional convolution operation on the multi-channel feature map to obtain a convolution output feature, perform overlapping segmentation on the convolution output feature to obtain a sub-feature map, and concatenate the sub-feature map with the convolution output feature and then perform a convolution operation to obtain the output feature.

[0063] like Figure 1 As shown, the feature overlapping segmentation graph convolutional network module includes a two-dimensional convolutional layer, a feature overlapping segmentation block and a convolution block. Among them, the two-dimensional convolutional layer is used to adjust the dimension of the multi-channel feature map and enhance its expression ability to obtain the convolution output feature. The feature overlapping segmentation block is used to perform overlapping segmentation on the convolution output feature, and construct a graph structure after splicing the obtained sub-feature map with the convolution output feature, and obtain the second feature after performing the graph convolution operation. The convolution block is used to perform a convolution operation on the second feature to obtain the output feature.

[0064] In this embodiment, the convolution output features are overlapped and segmented, the obtained sub-feature graphs are spliced ​​with the convolution output features to construct a graph structure, and the second feature is obtained after the graph convolution operation, as follows:

[0065] (3.1) The convolution output feature is a three-dimensional feature, expressed as ,in, Represent the time dimension, frequency dimension and channel dimension respectively.

[0066] A mask matrix of the same size as the convolution output feature is applied to the convolution output feature, and multiple boundary overlap retention window mask matrices are applied to the convolution output feature by element-by-element multiplication to perform boundary overlap segmentation in the time dimension and frequency dimension, ensuring that each sub-feature map retains local feature information and contains the overlapping parts between adjacent regions. The elements of the mask matrix are 1 or 0, indicating the retention and neglect of features, respectively.

[0067] (3.2) The time dimension and the frequency dimension are combined as the feature dimension. The sub-feature map and the convolution output feature are transformed in dimension and then concatenated along the feature dimension to obtain a two-dimensional feature map. , two-dimensional feature map Include nodes, and the dimension of node features is .

[0068] (3.3) Perform a pooling operation on the concatenated two-dimensional feature map to achieve dimensionality reduction.

[0069] The graph pooling operation is performed on the two-dimensional feature map , build a graph structure model on the feature dimension, convert the feature data into a graph, map the feature dimension features to the graph nodes, and reduce the number of nodes through the graph pooling layer to reduce the computational complexity. The details are as follows:

[0070] Calculate the projection score of each node, expressed as:

[0071] ;

[0072] in, represents the projection score of the nth node, , represents the nth node, Represents the projection vector.

[0073] Select the front with the largest projection score nodes are reserved, among which, , k represents the pooling rate, which is used to control the number of retained nodes. The eigenvalues ​​of the nodes are weighted to obtain The pooled feature map of nodes is expressed as:

[0074] ;

[0075] in, express The feature matrix of each node, The projection score vector of nodes, represents element-wise multiplication, Represents the activation function.

[0076] (3.4) Based on the , construct the graph structure and map its channel dimension features into graph nodes, and obtain nodes, and the number of dimensions of node features is Feature map , the feature map Enter the Graph Convolutional Network (GCN).

[0077] The graph convolutional network contains two convolutional layers, and the propagation between layers is expressed as:

[0078] ;

[0079] in, represents the feature matrix of the lth layer, represents the feature matrix of the l+1th layer, represents the adjacency matrix, , I represents the identity matrix, and Respectively and The degree matrix of represents the weight matrix of the linear transformation of the lth layer, represents a non-linear activation function.

[0080] After passing through the graph convolutional network, the output contains nodes, and the dimension of node features is The second characteristic .

[0081] (4) The embedding feature generation module is used to generate speech embedding features based on the integrated features.

[0082] Specifically, the multi-channel feature map and the output feature are residually connected to obtain an integrated feature input embedding feature generation module. The embedding feature generation module includes a time-frequency heterogeneous image block and a maximum image operation block.

[0083] Furthermore, the time-frequency heterogeneous graph block is used to construct graph structures in the time domain dimension and frequency domain dimension of the integrated features respectively, to obtain a time domain graph structure and a frequency domain graph structure, and to integrate the node information of the time domain graph structure and the frequency domain graph structure to obtain a time-frequency heterogeneous graph.

[0084] The maximum graph operation block includes two parallel branches. Each branch processes the time-frequency heterogeneous graph through a graph attention network, outputs branch features, performs maximum operation and node processing on the two branch features, and obtains speech embedding features.

[0085] Step 3: Obtain the speech authenticity probability based on the speech embedding feature, and compare the speech authenticity probability with the set threshold to determine whether the original speech is authentic. The details are as follows:

[0086] The speech embedding feature is mapped through a linear layer to obtain a prediction score vector , the two components of the prediction score vector represent the true speech prediction score and the deceptive speech prediction score respectively.

[0087] The real speech prediction score is subjected to a softmax operation to generate a speech authenticity probability.

[0088] The speech authenticity probability is compared with a set threshold. If the speech authenticity probability is greater than or equal to the set threshold, it is judged to be a real speech; otherwise, it is judged to be a deceptive speech.

[0089] Example 2

[0090] This embodiment provides an experimental application of the method described in Embodiment 1. The details are as follows:

[0091] After the original speech data is processed by the data enhancement module and the feature extraction module, a multi-channel feature map is obtained. , the characteristic shape is .

[0092] Multi-channel feature map Input feature overlapping segmentation map convolutional network module, through the two-dimensional convolution layer multi-channel feature map Perform dimension adjustment while maintaining convolution output features and multi-channel feature maps The number of channels is consistent, both are 64, and the characteristic shape of the convolution output feature is In this embodiment, the convolution kernel size of the two-dimensional convolution layer is , step size is 1, no padding above and below, and padding of 1 on the left and right.

[0093] The convolution output feature input feature overlap segmentation block performs overlapping segmentation. By applying a mask matrix of the same size as the convolution output feature to the convolution output feature, the four boundary overlap retention window mask matrices are applied to the convolution output feature by element-by-element multiplication to achieve boundary overlap segmentation in the time-frequency dimension. In this embodiment, the number of overlapping elements is 4, the number of sub-feature graphs is 4, and the feature shapes of the sub-feature graphs are respectively , , and .

[0094] After the sub-feature map and the convolution output feature are transformed in dimension, they are concatenated along the feature dimension to obtain five two-dimensional feature components including the feature dimension and the channel dimension. The feature shapes are , , , and , the characteristic shape of the two-dimensional feature is .

[0095] The concatenated two-dimensional features are modeled into a graph structure, and the feature dimension features are mapped into graph nodes. The resulting graph structure contains nodes, and the dimension of node features is . Calculate the projection score for each node , select the one with the highest score nodes to be reserved. , get the value containing The graph structure of nodes is constructed by element-wise multiplication. The feature values ​​of the nodes are weighted, and finally the nodes and the number of dimensions of node features is Pooled feature map.

[0096] The pooling feature map is modeled through the graph structure, and the channel dimension features are mapped to graph nodes. The resulting graph structure contains nodes, and the node feature dimension is Feature map , the feature map Enter graph convolutional networks.

[0097] After passing through the graph convolutional network, the output contains nodes, and the dimension of node features is The second characteristic .

[0098] The second feature The input convolution block undergoes dimension transformation, batch normalization layer, SeLU activation function, and convolution kernel size. , a two-dimensional convolutional layer with a stride of 1, no padding above and below, and padding of 1 on the left and right, the output features , the characteristic shape is .

[0099] The multi-channel feature map With output features Perform residual connection to obtain integrated features , the characteristic shape is .

[0100] Integration Features Input the embedding feature generation module to obtain the speech embedding feature ,in , represents the dimension of the embedded feature. Based on the speech embedded feature, the speech authenticity probability is obtained, and the speech authenticity probability is compared with the set threshold 0.5 to determine whether the original speech is real speech.

[0101] In this embodiment, the equal error rate (EER) and the minimum tandem detection cost function (min t-DCF) are used as evaluation indicators for the judgment results. Based on the above two indicators, the experimental results of this method are compared with the experimental results of voice anti-spoofing judgment under the Wav2vec-AASIST system, as shown in Table 1:

[0102] Table 1 Comparison results of EER and min t-DCF

[0103] min t-DCF EER (%) Wav2vec-AASIST 0.2134 1.06 This method 0.2054 0.76

[0104] As can be seen from Table 1, the EER and min t-DCF evaluation indicators of this method are improved by 28.30% and 3.75% respectively compared with the Wav2vec-AASIST system. This shows that this method effectively improves the accuracy and reliability of voice anti-spoofing judgment.

[0105] Example 3

[0106] This embodiment provides a voice anti-spoofing system based on a feature overlapping segmentation graph convolutional network, including:

[0107] The data acquisition module is used to acquire original voice data.

[0108] The embedding feature extraction module is used to input the original voice data into a pre-trained voice anti-spoofing network model to obtain voice embedding features.

[0109] The judgment module is used to obtain the speech authenticity probability based on the speech embedding feature, and compare the speech authenticity probability with a set threshold to judge whether the original speech is real speech.

[0110] Among them, the speech anti-spoofing network model includes a data enhancement module, a feature extraction module, a feature overlapping segmentation graph convolutional network module and an embedded feature generation module.

[0111] The data enhancement module is used to perform data enhancement operation on the original speech data to obtain original speech enhanced data.

[0112] The feature extraction module is used to extract multi-channel features in the original speech enhancement data to obtain a multi-channel feature map.

[0113] The feature overlapping segmentation map convolutional network module is used to perform a two-dimensional convolution operation on the multi-channel feature map to obtain a convolution output feature, perform overlapping segmentation on the convolution output feature to obtain a sub-feature map, and perform a convolution operation after splicing the sub-feature map with the convolution output feature to obtain the output feature.

[0114] The multi-channel feature map and the output feature are residually connected to obtain an integrated feature; the embedded feature generation module is used to generate a speech embedded feature based on the integrated feature.

[0115] Example 4

[0116] The present embodiment provides a speech anti-spoofing device based on a feature overlapping segmentation graph convolutional network, comprising a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the speech anti-spoofing method based on a feature overlapping segmentation graph convolutional network described in Example 1.

[0117] Example 5

[0118] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the voice anti-spoofing method based on the feature overlapping segmentation graph convolutional network described in Embodiment 1 are implemented.

[0119] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0120] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0121] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0122] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0123] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A voice anti-spoofing method based on feature overlapping segmentation graph convolutional network, characterized in that: include: Get the original voice data; Inputting the original speech data into a pre-trained speech anti-spoofing network model to obtain speech embedding features; Obtaining a speech authenticity probability based on the speech embedding feature, and comparing the speech authenticity probability with a set threshold to determine whether the original speech is a true speech; The speech anti-spoofing network model includes a data enhancement module, a feature extraction module, a feature overlapping segmentation graph convolutional network module and an embedded feature generation module; The data enhancement module is used to perform data enhancement operation on the original speech data to obtain original speech enhanced data; The feature extraction module is used to extract multi-channel features in the original speech enhancement data to obtain a multi-channel feature map; The feature overlapping segmentation graph convolutional network module is used to perform a two-dimensional convolution operation on the multi-channel feature graph to obtain a convolution output feature, perform overlapping segmentation on the convolution output feature to obtain a sub-feature graph, and perform a convolution operation after splicing the sub-feature graph with the convolution output feature to obtain an output feature; The multi-channel feature map and the output feature are residually connected to obtain an integrated feature; the embedded feature generation module is used to generate a speech embedded feature based on the integrated feature.

2. The voice anti-spoofing method based on feature overlapping segmentation graph convolutional network according to claim 1 is characterized in that: The data enhancement operation performed on the original speech data includes: introducing linear and nonlinear noise to perform noise adding processing on the original speech data.

3. The voice anti-spoofing method based on feature overlapping segmentation graph convolutional network according to claim 1 is characterized in that: The feature extraction module includes a self-supervision module and a residual network module; The self-supervision module includes a convolutional neural network and a transformer architecture, which is used to extract features from the original speech enhancement data to obtain a first feature; The residual network module includes multiple residual blocks, which are used to perform multi-channel feature extraction on the first feature to obtain a multi-channel feature map.

4. The voice anti-spoofing method based on feature overlapping segmentation graph convolutional network according to claim 1 is characterized in that: The feature overlapping segmentation graph convolutional network module includes a two-dimensional convolutional layer, a feature overlapping segmentation block and a convolution block; The two-dimensional convolution layer is used to adjust the dimension of the multi-channel feature map to obtain convolution output features; The feature overlapping segmentation block is used to perform overlapping segmentation on the convolution output feature, construct a graph structure after splicing the obtained sub-feature graph with the convolution output feature, and obtain the second feature after performing a graph convolution operation; The convolution block is used to perform a convolution operation on the second feature to obtain an output feature.

5. The voice anti-spoofing method based on feature overlapping segmentation graph convolutional network according to claim 4 is characterized in that: The overlapping segmentation of the convolution output features, the splicing of the obtained sub-feature graphs with the convolution output features to construct a graph structure, and the second feature is obtained after the graph convolution operation, including: The convolution output feature is a three-dimensional feature, expressed as ,in, Represent the time dimension, frequency dimension and channel dimension respectively; Applying a mask matrix of the same size as the convolution output feature to the convolution output feature, and applying multiple boundary overlap retention window mask matrices to the convolution output feature by element-by-element multiplication to perform boundary overlap segmentation in the time dimension and the frequency dimension, wherein the elements of the mask matrix are 1 or 0, respectively indicating retention and neglect of the feature; The time dimension and the frequency dimension are combined as the feature dimension, and the sub-feature map and the convolution output feature are transformed in dimension and then concatenated along the feature dimension to obtain a two-dimensional feature map. , two-dimensional feature map Include nodes, and the dimension of node features is ; Calculate the projection score of each node, expressed as: ； in, represents the projection score of the nth node, , represents the nth node, represents the projection vector; Select the front with the largest projection score nodes are reserved, among which, , k represents the pooling rate, and the obtained The pooled feature map of nodes is expressed as: ； in, express The feature matrix of each node, The projection score vector of nodes, represents element-wise multiplication, represents the activation function; Based on the , construct the graph structure and map its channel dimension features into graph nodes, and obtain nodes, and the number of dimensions of node features is Feature map , the feature map Input graph convolutional network; The graph convolutional network contains two convolutional layers, and the propagation between layers is expressed as: ； in, represents the feature matrix of the lth layer, represents the feature matrix of the l+1th layer, represents the adjacency matrix, , I represents the identity matrix, and Respectively and The degree matrix of represents the weight matrix of the linear transformation of the lth layer, represents a nonlinear activation function; After passing through the graph convolutional network, the output contains nodes, and the dimension of node features is The second characteristic .

6. The voice anti-spoofing method based on feature overlapping segmentation graph convolutional network according to claim 1, characterized in that: The embedding feature generation module includes a time-frequency heterogeneous graph block and a maximum graph operation block; The time-frequency heterogeneous graph block is used to construct graph structures in the time domain dimension and the frequency domain dimension of the integrated features respectively, to obtain a time domain graph structure and a frequency domain graph structure, and to integrate the node information of the time domain graph structure and the frequency domain graph structure to obtain a time-frequency heterogeneous graph; The maximum graph operation block includes two parallel branches; each branch processes the time-frequency heterogeneous graph through a graph attention network, outputs branch features, performs maximum operation and node processing on the two branch features, and obtains speech embedding features.

7. The voice anti-spoofing method based on feature overlapping segmentation graph convolutional network according to claim 1 is characterized in that: The method of obtaining the speech authenticity probability based on the speech embedding feature and comparing the speech authenticity probability with a set threshold to determine whether the original speech is a real speech includes: The speech embedding feature is mapped through a linear layer to obtain a prediction score vector , the two components of the prediction score vector represent the real speech prediction score and the deceptive speech prediction score respectively; The real speech prediction score is subjected to a softmax operation to generate a speech authenticity probability; The speech authenticity probability is compared with a set threshold. If the speech authenticity probability is greater than or equal to the set threshold, it is judged to be a real speech; otherwise, it is judged to be a deceptive speech.

8. A voice anti-spoofing system based on feature overlapping segmentation graph convolutional network, characterized in that: include: A data acquisition module, used to acquire original voice data; An embedding feature extraction module, used for inputting the original voice data into a pre-trained voice anti-spoofing network model to obtain voice embedding features; A judgment module, used to obtain a speech authenticity probability based on the speech embedding feature, and compare the speech authenticity probability with a set threshold to determine whether the original speech is a real speech; The speech anti-spoofing network model includes a data enhancement module, a feature extraction module, a feature overlapping segmentation graph convolutional network module and an embedded feature generation module; The data enhancement module is used to perform data enhancement operation on the original speech data to obtain original speech enhanced data; The feature extraction module is used to extract multi-channel features in the original speech enhancement data to obtain a multi-channel feature map; The feature overlapping segmentation graph convolutional network module is used to perform a two-dimensional convolution operation on the multi-channel feature graph to obtain a convolution output feature, perform overlapping segmentation on the convolution output feature to obtain a sub-feature graph, and perform a convolution operation after splicing the sub-feature graph with the convolution output feature to obtain an output feature; The multi-channel feature map and the output feature are residually connected to obtain an integrated feature; the embedded feature generation module is used to generate a speech embedded feature based on the integrated feature.

9. A voice anti-spoofing device based on feature overlapping segmentation graph convolutional network, characterized in that: including processor and storage medium; The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the speech anti-spoofing method based on the feature overlapping segmentation graph convolutional network as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the speech anti-spoofing method based on the feature overlapping segmentation graph convolutional network described in any one of claims 1 to 7 are implemented.

Images