Lightweight forged voice detection method and system based on mixed pooling strategy

Abstract

The invention provides a lightweight forged voice detection method and system based on a mixed pooling strategy, and belongs to the technical field of voice processing and biological recognition. Extracting time-frequency information from the original voice data through convolution pooling operation; extracting high-level feature representation from the time-frequency information based on a mixed pooling method, namely extracting low-frequency features by adopting a maximum pooling method and extracting high-frequency features by adopting an average pooling method; performing feature splicing on the low-frequency features and the high-frequency features to obtain advanced feature representation; performing statistical calculation on the high-level feature representation according to the directions of the frequency domain and the time sequence to obtain a high-level feature statistical result; and detecting the forged voice based on the full connection layer and outputting a detection result. According to the method, on the basis of avoiding the loss of feature information, the calculation amount and the calculation complexity during forged voice detection can be reduced, and the capability of coping with unknown attacks is high; and meanwhile, the accuracy of detecting the forged voice is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of speech processing and biometric recognition, and in particular relates to a lightweight forged speech detection method and system based on a hybrid pooling strategy. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of digital information sharing technology and digital information editing technology on social networks, intelligent voice technology has been widely used in various devices and scenarios, greatly improving people's convenience in life. However, this technology also faces severe security challenges, especially the increasing threat of forged voice attacks. Attackers can bypass the authentication of intelligent voice systems by using technologies such as recording playback, speech synthesis and speech conversion, exposing personal information and privacy to greater risks. Therefore, forged voice detection has become a critical task, which aims to accurately distinguish between real human voices and fake voices synthesized by electronic devices.

[0004] Currently, there are many studies on forged speech detection that show that spectrum-based features have good discrimination capabilities, and that using different frequency band features has different effects on improving the performance of deception detection. Therefore, the front-end feature extraction module used for feature extraction in most detection models considers features from the power spectrum and uses Fourier transform to transform the signal from the time domain to the frequency domain.

[0005] However, there are two main methods used in the existing forged voice detection, both of which have some technical problems, specifically:

[0006] (1) An existing method for detecting fake speech is to use a single pooling method in the entire frequency range to detect fake speech. The disadvantage of this method is that it is easy to cause overfitting. For example, when a single pooling method is used in the entire frequency range, high-frequency band features are easy to introduce noise features, which can easily lead to overfitting, thereby causing inaccurate fake speech detection results.

[0007] (2) Another existing method for detecting fake speech is to detect fake speech by using a single pooling method for the low-frequency band and discarding the high-frequency band data. The disadvantage of this method is that it discards the high-frequency data, making the detection of fake features incomplete, which also leads to inaccurate calculation results. Summary of the invention

[0008] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a lightweight forged speech detection method and system based on a hybrid pooling strategy, which can reduce the amount of calculation and complexity of forged speech detection while avoiding the loss of feature information, and has strong ability to deal with unknown attacks; at the same time, it significantly improves the accuracy of forged speech detection.

[0009] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0010] A first aspect of the present invention provides a lightweight forged speech detection method based on a hybrid pooling strategy.

[0011] A lightweight forged speech detection method based on a hybrid pooling strategy includes:

[0012] Obtaining original voice data to be detected;

[0013] Performing a preprocessing operation on the original speech data and extracting time-frequency information using a convolutional pooling operation;

[0014] Extracting high-level feature representation from the time-frequency information based on a hybrid pooling method, specifically: extracting low-frequency features using a maximum pooling method, and extracting high-frequency features using an average pooling method; performing feature concatenation on the obtained low-frequency features and high-frequency features to obtain high-level feature representation;

[0015] Normalizing the high-level feature representation, and performing statistical calculations on the high-level feature representation in the frequency domain and the time series in turn to obtain high-level feature statistical results;

[0016] The obtained high-level feature statistics are input into the fully connected layer to detect the lightweight forged speech and output the detection results.

[0017] Furthermore, a preprocessing operation is performed on the original speech data, and a convolution pooling operation is used to extract time-frequency information, including: performing a fast Fourier transform on the original speech data to obtain corresponding frequency domain information; dividing the obtained frequency domain information into a specified number of frequency band information, and performing data processing on the frequency band information to extract key features of the frequency domain information; and then, obtaining the time-frequency information based on one-dimensional convolution.

[0018] Furthermore, the frequency domain range of the frequency domain information is set according to the Nyquist sampling theorem. Specifically, the frequency domain range of the frequency domain information is half of the sampling frequency, and the obtained frequency domain range is used as the full-band frequency domain information.

[0019] Furthermore, the maximum pooling method is used to extract low-frequency features, and the average pooling method is used to extract high-frequency features, including: dividing the number of frequency bands S of the time-frequency information into low-frequency features and high-frequency features according to the frequency bands. and high frequency Two parts, when If it is not an integer, round it up; in the low frequency band The maximum pooling method is used to extract the low-frequency features after pooling, and in the high-frequency band The average pooling method is used to extract the high-frequency features after pooling.

[0020] Furthermore, after the pooling operation is completed, the pooling results of the obtained low-frequency features and the pooling results of the high-frequency features are re-joined into a full-band full-frequency feature coding vector, and the full-frequency feature coding vector is used as a high-level feature representation.

[0021] Furthermore, the high-level feature representation is statistically calculated in the frequency domain and time series directions, including: first, the high-level feature representations of different channels in the frequency direction are encoded according to multiple attention weights of equal value, and the mean features of the frequency domain are obtained in the channel direction and the time series direction in turn to obtain new feature representations of different channels in the time series direction; then, the feature representations of different channels in the time series direction are statistically analyzed according to the distribution characteristics of the feature representations of the channels in the time series, so as to obtain the discrete distribution characteristics of the high-dimensional feature representations of different channels in the time series direction.

[0022] Furthermore, the obtained detection results are compared with the actual results based on the loss function to obtain the difference between the actual value and the predicted value, which is used to update the model parameters.

[0023] A second aspect of the present invention provides a lightweight forged speech detection system based on a hybrid pooling strategy.

[0024] A lightweight forged speech detection system based on a hybrid pooling strategy, including:

[0025] The data acquisition module is configured to: obtain original voice data to be detected;

[0026] The time-frequency information acquisition module is configured to: perform a preprocessing operation on the original speech data and extract the time-frequency information by using a convolution pooling operation;

[0027] The high-level feature statistics module is configured to: extract high-level feature representation from the time-frequency information based on a hybrid pooling method, specifically: extract low-frequency features using a maximum pooling method, and extract high-frequency features using an average pooling method; perform feature concatenation on the obtained low-frequency features and high-frequency features to obtain high-level feature representation; normalize the high-level feature representation, and perform statistical calculations on the high-level feature representation in the frequency domain and the time series in turn to obtain high-level feature statistical results;

[0028] The forged speech detection module is configured to: input the obtained high-level feature statistical results into the fully connected layer, detect the lightweight forged speech and output the detection results.

[0029] The third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in the lightweight forged speech detection method based on a hybrid pooling strategy as described in the first aspect of the present invention.

[0030] The fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the lightweight forged speech detection method based on the hybrid pooling strategy as described in the first aspect of the present invention are implemented.

[0031] One or more of the above technical solutions have the following beneficial effects:

[0032] (1) The present invention extracts high-level feature representations from time-frequency information based on a hybrid pooling method, namely: the maximum pooling method is used to extract low-frequency features, and the average pooling method is used to extract high-frequency features; the low-frequency features and the high-frequency features are spliced ​​to obtain high-level feature representations. By dividing the full-band spectrogram into low-frequency and high-frequency parts, different pooling methods are used for the two parts of features, and then the two parts of features are re-spliced ​​to form new full-band features. Therefore, the present invention is able to extract feature representations more effectively, thereby significantly improving the accuracy of detecting forged speech.

[0033] (2) The present invention adjusts the position of the compression excitation module to before the residual network, so that the training of channel attention takes precedence over the learning of spectral features during the training process; at the same time, the dependency relationship between the spectral features of different channels is modeled and a new loss function is designed to make the sample distribution more aggregated. Therefore, the present invention can effectively reduce the amount of calculation and complexity of calculation during forged speech detection, thereby improving the ability to deal with unknown forged speech attacks.

[0034] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0036] Figure 1 This is a flow chart of a lightweight forged speech detection method based on a hybrid pooling strategy in Embodiment 1 of the present invention.

[0037] Figure 2 Schematic diagram of the structural relationship of the units involved in the feature extraction process in the first embodiment of the present invention.

[0038] Figure 3This is a flow chart of a method for obtaining full-frequency domain features in Embodiment 1 of the present invention. DETAILED DESCRIPTION

[0039] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0040] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0041] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0042] The overall idea proposed by the present invention is as follows: the present invention proposes a lightweight forged speech detection method based on a hybrid pooling strategy, which extracts primary features of the original speech through a spectrum information extraction module; obtains high-level feature representations with channel information dependencies through a residual network feature extraction module with compression excitation pre-position; at the same time, a hybrid pooling method is adopted, by using maximum pooling for the low-frequency band and average pooling for the high-frequency band, and then concatenating feature codes to extract high-level feature representations of the entire frequency band; finally, a new loss function is designed, and end-to-end model optimization is performed through a single classification learning module.

[0043] Embodiment 1

[0044] This embodiment discloses a lightweight forged speech detection method based on a hybrid pooling strategy.

[0045] like Figure 1 As shown, a lightweight forged speech detection method based on a hybrid pooling strategy includes:

[0046] Step S1, obtaining original speech data to be detected;

[0047] Step S2, performing a preprocessing operation on the original speech data, and extracting time-frequency information using a convolutional pooling operation;

[0048] Step S3, extracting high-level feature representation from the time-frequency information based on a hybrid pooling method, specifically: extracting low-frequency features using a maximum pooling method, and extracting high-frequency features using an average pooling method; performing feature concatenation on the obtained low-frequency features and high-frequency features to obtain a high-level feature representation;

[0049] Step S4, normalizing the high-level feature representation, and performing statistical calculations on the high-level feature representation in the frequency domain and the time series in turn to obtain high-level feature statistical results;

[0050] Step S5: Input the obtained high-level feature statistics into the fully connected layer, detect the lightweight forged speech and output the detection result.

[0051] Based on the above steps, the present invention can reduce the amount of calculation and complexity of calculation when detecting forged voices on the basis of avoiding the loss of feature information, and has a strong ability to deal with unknown attacks; at the same time, it significantly improves the accuracy of detecting forged voices. To facilitate the understanding of the technical solution of the present invention, the specific implementation steps of the technical solution of the present invention are further explained and illustrated below.

[0052] Step S1: Obtain original speech data to be detected.

[0053] The original speech data to be tested is obtained by using the data in the LA dataset in the ASVSpoof2019 corpus, which is commonly used for forged speech detection. Among them, the speech data in the LA dataset in the ASVSpoof2019 corpus are all obtained based on the VCTK corpus. The VCTK corpus collects the real speech of 107 speakers, including 46 males and 61 females. All the real speech uses the same recording configuration and is not interfered by channel and background noise.

[0054] Step S2: preprocessing the original speech data and extracting time-frequency information using convolutional pooling.

[0055] Perform fast Fourier transform on the original speech data to obtain the corresponding frequency domain information; divide the obtained frequency domain information into a specified number of frequency band information, and perform data processing on the frequency band information to extract the key features of the frequency domain information; wherein the data processing process includes performing convolution, normalization, nonlinear transformation and maximum pooling on the frequency domain information on each frequency band. Subsequently, the time-frequency information is obtained based on one-dimensional convolution. The frequency domain range of the frequency domain information is set according to the Nyquist sampling theorem. Specifically, the frequency domain range of the frequency domain information is set to half of the sampling frequency, and the set frequency domain range is used as the full-band frequency domain information. The initial time-frequency feature information is obtained through one-dimensional convolution. This process can effectively reduce the number of model parameters and the amount of calculation through convolution and pooling operations.

[0056] Step S2-1, using a Sinc convolutional layer that performs time-domain convolution for front-end feature learning through a series of parameterized Sinc functions aligned with rectangular bandpass filters.

[0057] In the frequency domain, a low-pass filter can be represented by a rectangular function, while a band-pass filter is written as the difference between two low-pass filters, that is:

[0058]

[0059] Among them, G() represents the bandpass filter function in the frequency domain, f represents the sampling frequency of the bandpass filter in the frequency domain, f1 and f2 represent the starting frequency and cutoff frequency of the bandpass filter respectively, and rect() represents a low-pass filter.

[0060] Furthermore, the Sinc function is defined as:

[0061] Sinc(πx)=sin(πx) / πx;

[0062] Where x represents the equation variable. Therefore, the bandpass filter function in the time domain can be expressed as:

[0063] g(n,f1,f2)=2f2Sinc(2πf2)-2f1Sinc(2πf1);

[0064] Among them, g() represents the bandpass filter function in the time domain, and n represents the sampling frequency of the bandpass filter in the time domain.

[0065] Step S2-2: These bandpass filter functions correspond to rectangular bandpass filters, and the starting frequency in each filter group is determined by the frequency band range and the target number of divided frequency bands.

[0066] Decompose all frequencies into S non-overlapping equivalent parts, that is, divide them into S sub-bands, and the starting frequency of the i-th sub-band is f 1-i =(f / 2S)(i-1), the cutoff frequency is f 2-i =i*(f / 2S), the bandwidth is f / 2S; where i=1,2,...,S. In this embodiment, all frequencies are decomposed into 70 non-overlapping equivalent parts, that is, divided into 70 sub-bands, i=1,2,...,70; where the starting frequency of the i-th sub-band is f 1-i =(4000 / 35)(i-1)Hz, cut-off frequency is f 2-i =i*(4000 / 35)Hz, bandwidth is 4000 / 35Hz. Thus, a feature encoding with a dimensional space of [70,128000] is obtained, where 128000 is the number of samples obtained with a sampling frequency of 16k and a continuous acquisition time of 8 seconds.

[0067] Step S2-3, minimize the calculation through the maximum pooling operation to obtain the simplified spectral features, and then batch normalize the resulting feature map.

[0068] First, the result of the above steps is subjected to the conv1D(70,7,129) operation to obtain a feature encoding with a dimension space of [70,18268]; wherein conv1D(70,7,129) represents a one-dimensional convolution operation, which means that the original speech data is subjected to a convolution operation after Fourier transformation to reduce the data volume; 70 represents dividing the data into 70 equal parts in the frequency direction; 7 represents a skip operation. In this embodiment, the original data volume is 128000. After this operation, the data volume is reduced to 18286; 129 represents the convolution kernel size. Specifically, through the Sinc layer network, channel information is added to obtain the feature map FM∈R CxSxT ; Where C, S, and T represent the number of channels, the number of features in the spectral dimension, and the number of features in the time dimension, respectively; at this time, the mapping space dimension is [C, S, T] = [1, 70, 128000].

[0069] Subsequently, the simplified spectral features are obtained through maximum pooling Maxpool-2D(1,3) to minimize the calculation, and batch normalization is performed on them. After nonlinear transformation by the activation function, a spectral feature mapping space with a final dimension size of [C, S, T] = [1, 70, 6089] is obtained.

[0070] Step S3, extracting high-level feature representation from time-frequency information based on a hybrid pooling method, specifically: using the maximum pooling method to extract low-frequency features, and using the average pooling method to extract high-frequency features; and concatenating the obtained low-frequency features and high-frequency features to obtain high-level feature representation.

[0071] After extracting the high-level feature representation of the original speech information, the three-dimensional representation of the time-frequency features of different channels is obtained through the pre-placed residual network with a channel attention compression excitation module, that is: the high-level feature representation of the original speech information is extracted through the residual neural network module with channel attention, where the residual neural network module includes eight basic units such as compression excitation, convolution, normalization, activation and pooling. Each basic unit can perform the same data processing operation on the input feature information, and the interdependent feature information at the channel level is obtained through the pre-placed residual network with a channel attention compression excitation module; it is summed with the recorded input feature information and then processed by hybrid pooling to obtain a feature coding vector. After the eight basic units are executed in the unit configuration order of the pre-placed compression excitation module, the high-level feature representation of the original speech is obtained. This process strengthens the dependency between the feature information between channels through the pre-placed compression excitation module, and at the same time, the convolution pooling operation reduces the number of model parameters and the amount of calculation.

[0072] Furthermore, if Figure 2As shown, this embodiment can significantly improve the effective use of full-frequency range features by placing a channel attention compression excitation module before the residual coding network module and extracting high-level feature representation of speech information from the obtained spectral information according to the hybrid pooling method; thus, the dependency relationship between feature data between channels can be obtained in advance, which is conducive to focusing on more influential channel features, thereby effectively improving the detection accuracy. Among them, the working principles of the compression excitation module and the residual coding network module are as follows:

[0073] For the compressed excitation module, the current feature dimension size is [C, S, T], and the [S, T] dimension of the feature map is average pooled to obtain a channel weight map with a dimension size of [C, 1, 1]. This is input into a 2-layer perceptron with nonlinear changes to generate a final channel weight map with a dimension size of [C, 1, 1]. The weight map is applied to the spectral feature [C, S, T] to obtain a spectral feature with a dimension size of [C, S, T] and channel dependency.

[0074] The residual coding network module is composed of a compression excitation module, a convolution layer, a normalization layer, and an activation layer. The specific design sequence is compression excitation module → convolution layer 1 → normalization layer 1 → activation layer 1 → convolution layer 2 → normalization layer 2 → the output result of the spectrum information extraction step is summed with the output result of the normalization layer 2 and then input to the activation layer 2 to complete a set of residual coding network cycles. Here, this embodiment uses 5 sets of residual coding network cycles to obtain high-level feature mapping results.

[0075] For the hybrid pooling method, the pooling unit extracts the distribution characteristics of the high-level feature representation of the original speech information according to the initial time-frequency feature information, and adopts the hybrid pooling method to obtain a new feature representation, including using the maximum pooling method to extract low-frequency features and using the average pooling method to extract high-frequency features, that is, dividing the frequency band number S of the time-frequency information into low-frequency and high-frequency features according to the frequency band. and high frequency Two parts, when If it is not an integer, it is rounded up; according to the high-level feature representation of the original speech information, the low-frequency band features are dense and the high-frequency band features are sparse, and the recorded input feature information is combined with the feature information processed by the neural network to obtain the low-frequency band feature. The maximum pooling method is used to extract the low-frequency features after pooling, and in the high-frequency band The average pooling method is used to extract the high-frequency features after pooling. After the pooling operation is completed, the pooling results of the low-frequency features and the high-frequency features are re-joined into a full-band full-frequency feature encoding vector, and the full-frequency feature encoding vector is used as a high-level feature representation, that is, a new full-frequency domain feature. Figure 3 shown.

[0076] Furthermore, in this embodiment, the entire feature extraction process is implemented based on the residual coding network module. Five groups of residual coding network cycles are used above, and the specific implementation steps are as follows:

[0077] Step S3-1, the residual coding network module performs the first cycle: execute the compression excitation module, the current feature dimension size is [1,70,6089], average pooling is performed on the [70,6089] dimension of the feature map, and a channel weight map with a dimension size of [1,1,1] is obtained; it is input into a 2-layer perceptron with nonlinear changes to generate a final channel weight map with a dimension size of [1,1,1]; the weight map is applied to the spectral feature [1,70,6089] to obtain a spectral feature with a dimension size of [1,70,6089] and a channel dependency.

[0078] The conv2D((2,3),1,32) operation is performed, and batch normalization is performed on it. After the nonlinear transformation of the activation function, the spectral feature mapping space with a final dimension size of [32,70,6089] is obtained. The conv2D((2,3),32,32) operation is performed, and batch normalization is performed on it. After the nonlinear transformation of the activation function, the spectral feature mapping space with a final dimension size of [32,70,6089] is obtained. Through the hybrid pooling method, the Maxpool-2D(1,3) operation is performed on the 0-35 frequency bands in the frequency band direction, and the Avgpool-2D(1,3) operation is performed on the 36-70 frequency bands, and then they are re-spliced ​​into the full-band feature encoding, and the mapping space dimension size is [32,70,2029]; where Maxpool-2D() represents the maximum pooling operation, and Avgpool-2D() represents the average pooling operation.

[0079] Step S3-2, the residual coding network module performs a second cycle: execute the compression excitation module, the current feature dimension size is [32,70,2029], average pooling is performed on the [70,2029] dimension of the feature map, and a channel weight map with a dimension size of [32,1,1] is obtained; it is input into a 2-layer perceptron with nonlinear changes to generate a final channel weight map with a dimension size of [32,1,1]. The weight map is applied to the spectral feature [32,70,2029] to obtain a spectral feature with a dimension size of [32,70,2029] and channel dependency.

[0080] Perform conv2D((2,3),32,32) operation and perform batch normalization on it. After nonlinear transformation of activation function, the final spectral feature mapping space with dimension size [32,70,2029] is obtained. Perform conv2D((2,3),32,32) operation and perform batch normalization on it. After nonlinear transformation of activation function, the final spectral feature mapping space with dimension size [32,70,2029] is obtained. Through hybrid pooling method, perform Maxpool-2D(1,3) operation on 0-35 frequency bands in frequency band direction, perform Avgpool-2D(1,3) operation on 36-70 frequency bands, and then re-splice them into full-band feature encoding. The mapping space dimension size is [32,70,676].

[0081] Step S3-3, the residual coding network module performs a third cycle: execute the compression excitation module, the current feature dimension size is [32,70,676], average pooling is performed on the [70,676] dimension of the feature map, and a channel weight map with a dimension size of [32,1,1] is obtained; it is input into a 2-layer perceptron with nonlinear changes to generate a final channel weight map with a dimension size of [32,1,1]; the weight map is applied to the spectral feature [32,70,676] to obtain a spectral feature with a dimension size of [32,70,676] and channel dependency.

[0082] Perform conv2D((2,3),32,24) operation and perform batch normalization on it. After nonlinear transformation of activation function, the final spectral feature mapping space with dimension size [24,70,676] is obtained. Perform conv2D((2,3),24,24) operation and perform batch normalization on it. After nonlinear transformation of activation function, the final spectral feature mapping space with dimension size [24,70,676] is obtained. Through hybrid pooling method, perform Maxpool-2D(1,3) operation on 0-35 frequency bands in frequency band direction, perform Avgpool-2D(1,3) operation on 36-70 frequency bands, and then re-splice them into full-band feature encoding, and the mapping space dimension size is [24,70,225].

[0083] Step S3-4, the residual coding network module performs the fourth cycle: execute the compression excitation module, the current feature dimension size is [24,70,225], and the [70,225] dimension of the feature map is average pooled to obtain a channel weight map with a dimension size of [24,1,1]; input it into a 2-layer perceptron with nonlinear changes to generate a final channel weight map with a dimension size of [24,1,1]; apply the weight map to the spectral feature [24,70,225] to obtain a spectral feature with a dimension size of [24,70,225] and channel dependency.

[0084] Perform conv2D((2,3),24,24) operation and perform batch normalization on it. After nonlinear transformation of activation function, the final spectral feature mapping space with dimension size [24,70,225] is obtained. Perform conv2D((2,3),24,24) operation and perform batch normalization on it. After nonlinear transformation of activation function, the final spectral feature mapping space with dimension size [24,70,225] is obtained. Through hybrid pooling method, perform Maxpool-2D(1,3) operation on 0-35 frequency bands in frequency band direction, perform Avgpool-2D(1,3) operation on 36-70 frequency bands, and then re-splice them into full-band feature encoding, and the mapping space dimension size is [24,70,75].

[0085] Step S3-5, the residual coding network module performs the fifth cycle: execute the compression excitation module, the current feature dimension size is [24,70,75], average pooling is performed on the [70,75] dimension of the feature map, and a channel weight map with a dimension size of [24,1,1] is obtained; it is input into a 2-layer perceptron with nonlinear changes to generate a final channel weight map with a dimension size of [24,1,1]; the weight map is applied to the spectral feature [24,70,75] to obtain a spectral feature with a dimension size of [24,70,75] and channel dependency.

[0086] Perform conv2D((2,3),24,24) operation and perform batch normalization on it. After nonlinear transformation of activation function, the final spectral feature mapping space with dimension size [24,70,75] is obtained. Perform conv2D((2,3),24,24) operation and perform batch normalization on it. After nonlinear transformation of activation function, the final spectral feature mapping space with dimension size [24,70,75] is obtained. Through hybrid pooling method, perform Maxpool-2D(1,3) operation on 0-35 frequency bands in frequency band direction, perform Avgpool-2D(1,3) operation on 36-70 frequency bands, and then re-splice them into full-band feature encoding, and the mapping space dimension size is [24,70,25].

[0087] This embodiment achieves lightweighting of the forged speech detection model by compressing the model structure and model parameters, including: model structure compression, that is, reducing the size of the model by changing the model structure, reducing storage and transmission costs; model parameter compression, that is: through pooling operations, reducing the dimension of feature representation, reducing the number of parameters in the network, thereby reducing the amount of model calculation.

[0088] Step S4: normalize the high-level feature representation, and perform statistical calculations on the high-level feature representation in the frequency domain and time series directions in turn to obtain high-level feature statistical results.

[0089] The high-level feature representation is statistically calculated in the frequency domain and time series direction, including: first, the high-level feature representation of different channels in the frequency direction is calculated according to the equal value N S Attention weight encoding; dot product is taken between the attention weight encoding and the spectrum feature in the channel direction and the time series direction, and the mean feature in the frequency domain is obtained to obtain new feature representations of different channels in the time series direction. Subsequently, the feature representations of different channels in the time series direction are statistically analyzed according to the distribution characteristics of the feature representations of the channels in the time series, and the discrete distribution characteristics of the high-dimensional feature representations of different channels in the time series direction are obtained. This can be achieved by the following steps:

[0090] Step S4-1, calculating the average value in the frequency domain.

[0091] Assumption: W=(w1,w2,...,w S ),w1=w2=...=w S ,

[0092] Where W represents the weight vector, w S Indicates the weight value assigned to the Sth frequency band. The statistical results WX in the frequency domain are solved in time sequence on each channel. S , to obtain a feature mapping space X with a dimension size of [24,25] T .

[0093] Step S4-2: Solve the feature distribution on each channel along the time series direction to obtain the feature distribution mapping on the channel, that is:

[0094]

[0095] F C =(σ1,σ2,...,σ C );

[0096] in, represents the average value of the feature along the time series direction, T represents the number of features along the time series direction, x i represents the i-th feature along the time series direction, σ C represents the distribution of features along the time series direction on the Cth channel; σ represents the feature distribution on each channel, F C represents the feature distribution map on the channel. Thus, we get a feature mapping space F with a dimension size of

[24] C .

[0097] Step S5: Input the obtained high-level feature statistics into the fully connected layer, detect the lightweight forged speech and output the detection result.

[0098] The statistical feature encoding on the channel is input into the fully connected layer to obtain the predicted value generated by the model, which is applied to the optimized loss function and compared with the actual result to obtain the difference between the actual value and the predicted value, which is used to update the model parameters.

[0099] Furthermore, the optimized loss function is designed as follows:

[0100]

[0101] Γ=(τ1,τ2,τ3,...,τ N );

[0102] in, represents the normalized embedding features, represents the normalized weight vector of the true class, is defined as the classification score of the i-th sample; y i ∈{0,1} is the label of the i-th sample, y i = 0 means the true class, y i =1 indicates a deception attack class. m0 and m1 correspond to the boundary values ​​of the true class and the deception class respectively; where m0, m1∈[-1,1], and m0>m1. In this embodiment, in order to facilitate the determination of the interval area for distinguishing the true class from the deception class, the boundary value of the true class is set to m0=0.5, and the boundary value of the deception class is set to m1=0.2. α represents the scaling factor, which is set to 20; N represents the batch size, which is set to 12. τ i Represents the sample weight, when τ i When τ is 1, the sample participates in the update process of model training; when τ i When it is 0, the sample does not participate in the update process of model training. The sample weight value is determined by the sample category and the classification score of the sample. Specifically, when the classification score of the sample is Less than 0.2, and the sample belongs to the deception attack class (y i =1), the sample weight τ i is 0; otherwise the sample weight τi is 1. Γ represents the sample weight τ in each training batch i The vector composed of; var(Γ) represents the sample distribution participating in the gradient update in each batch, var(Γ)∈[0,0.25]. This embodiment can achieve higher efficiency, stronger generalization performance and more accurate recognition rate forged speech detection through the designed new loss function; at the same time, the loss function can further reduce the adverse effects of the imbalanced data set, making the distribution of the detection results more compact, providing a double guarantee for improving the generalization ability and detection ability of the model.

[0103] After finally obtaining the trained network model, the network model is used as a classifier to distinguish real speech from fraudulent speech, and the real speech and forged speech are classified according to the numerical results in the output of the fully connected layer. Specifically: the obtained discrete distribution features are input into the fully connected layer to obtain the predicted score of the output sample classification, the predicted score and the true value are input into the loss function, the loss value is obtained through the loss function, and the next round of model training parameters are updated through back propagation and gradient descent. In this process, the loss value obtained by the loss function is updated along with the model training and parameter update to obtain a compact forged speech detection result.

[0104] Embodiment 2

[0105] This embodiment discloses a lightweight forged speech detection system based on a hybrid pooling strategy.

[0106] A lightweight forged speech detection system based on a hybrid pooling strategy, including:

[0107] The data acquisition module is configured to: obtain original voice data to be detected;

[0108] The time-frequency information acquisition module is configured to: perform a preprocessing operation on the original speech data and extract the time-frequency information by using a convolution pooling operation;

[0109] The high-level feature statistics module is configured to: extract high-level feature representation from the time-frequency information based on a hybrid pooling method, specifically: extract low-frequency features using a maximum pooling method, and extract high-frequency features using an average pooling method; perform feature concatenation on the obtained low-frequency features and high-frequency features to obtain high-level feature representation; normalize the high-level feature representation, and perform statistical calculations on the high-level feature representation in the frequency domain and the time series in turn to obtain high-level feature statistical results;

[0110] The forged speech detection module is configured to: input the obtained high-level feature statistical results into the fully connected layer, detect the lightweight forged speech and output the detection results.

[0111] Embodiment 3

[0112] The purpose of this embodiment is to provide a computer-readable storage medium.

[0113] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the lightweight forged speech detection method based on a hybrid pooling strategy as described in the first embodiment of the present disclosure.

[0114] Embodiment 4

[0115] The purpose of this embodiment is to provide an electronic device.

[0116] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in the lightweight forged speech detection method based on a hybrid pooling strategy as described in the first embodiment of the present disclosure are implemented.

[0117] The steps involved in the apparatuses of the above embodiments 2, 3 and 4 correspond to the method embodiment 1, and the specific implementation methods can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0118] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0119] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A lightweight forged speech detection method based on a hybrid pooling strategy, characterized in that: include: Obtaining original voice data to be detected; Performing a preprocessing operation on the original speech data and extracting time-frequency information using a convolutional pooling operation; Extracting high-level feature representation from the time-frequency information based on a hybrid pooling method, specifically: extracting low-frequency features using a maximum pooling method, and extracting high-frequency features using an average pooling method; performing feature concatenation on the obtained low-frequency features and high-frequency features to obtain high-level feature representation; Normalizing the high-level feature representation, and performing statistical calculations on the high-level feature representation in the frequency domain and the time series in turn to obtain high-level feature statistical results; The obtained high-level feature statistics are input into the fully connected layer to detect the lightweight forged speech and output the detection results.

2. The lightweight forged speech detection method based on hybrid pooling strategy as claimed in claim 1 is characterized in that: A preprocessing operation is performed on the original speech data, and a convolution pooling operation is used to extract time-frequency information, including: performing a fast Fourier transform on the original speech data to obtain corresponding frequency domain information; dividing the obtained frequency domain information into a specified number of frequency band information, and performing data processing on the frequency band information to extract key features of the frequency domain information; then, obtaining the time-frequency information based on a one-dimensional convolution.

3. The lightweight forged speech detection method based on hybrid pooling strategy as claimed in claim 2 is characterized in that: The frequency domain range of the frequency domain information is set according to the Nyquist sampling theorem. Specifically, the frequency domain range of the frequency domain information is half of the sampling frequency, and the obtained frequency domain range is used as the full-band frequency domain information.

4. The lightweight forged speech detection method based on hybrid pooling strategy as claimed in claim 1, characterized in that: The maximum pooling method is used to extract low-frequency features, and the average pooling method is used to extract high-frequency features, including: dividing the frequency band number S of the time-frequency information into low-frequency features and high-frequency features according to the frequency bands. and high frequency Two parts, when If it is not an integer, round it up; in the low frequency band The maximum pooling method is used to extract the low-frequency features after pooling, and in the high-frequency band The average pooling method is used to extract the high-frequency features after pooling.

5. The lightweight forged speech detection method based on hybrid pooling strategy as claimed in claim 4 is characterized in that: After the pooling operation is completed, the pooling results of the obtained low-frequency features and the pooling results of the high-frequency features are re-joined into a full-band full-frequency feature coding vector, and the full-frequency feature coding vector is used as a high-level feature representation.

6. The lightweight forged speech detection method based on hybrid pooling strategy as claimed in claim 1, characterized in that: The high-level feature representation is statistically calculated in the frequency domain and time series directions, including: first, the high-level feature representations of different channels in the frequency direction are encoded according to multiple attention weights of equal value, and the mean features of the frequency domain are obtained in the channel direction and the time series direction in turn to obtain new feature representations of different channels in the time series direction; then, the feature representations of different channels in the time series direction are statistically analyzed according to the distribution characteristics of the feature representations of the channels in the time series, so as to obtain the discrete distribution characteristics of the high-dimensional feature representations of different channels in the time series direction.

7. The lightweight forged speech detection method based on hybrid pooling strategy as claimed in claim 1 is characterized in that: Also includes: The obtained detection results are compared with the actual results based on the loss function to obtain the difference between the actual value and the predicted value, which is used to update the model parameters.

8. A lightweight forged speech detection system based on a hybrid pooling strategy, characterized in that: include: The data acquisition module is configured to: obtain original voice data to be detected; The time-frequency information acquisition module is configured to: perform a preprocessing operation on the original speech data and extract the time-frequency information by using a convolution pooling operation; The high-level feature statistics module is configured to: extract high-level feature representation from the time-frequency information based on a hybrid pooling method, specifically: extract low-frequency features using a maximum pooling method, and extract high-frequency features using an average pooling method; perform feature concatenation on the obtained low-frequency features and high-frequency features to obtain high-level feature representation; normalize the high-level feature representation, and perform statistical calculations on the high-level feature representation in the frequency domain and the time series in turn to obtain high-level feature statistical results; The forged speech detection module is configured to: input the obtained high-level feature statistical results into the fully connected layer, detect the lightweight forged speech and output the detection results.

9. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps in the lightweight forged speech detection method based on a hybrid pooling strategy as described in any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the lightweight forged speech detection method based on the hybrid pooling strategy as described in any one of claims 1 to 7 are implemented.

Images