Digital audio tampering passive detection method and device based on power grid frequency deep and shallow feature fusion

Abstract

The present application relates to a kind of digital audio tampering passive detection method and device based on grid frequency shallow layer feature fusion, first to the audio data to be detected is handled to obtain the ENF phase of grid frequency (ENF) component and extract the instantaneous frequency of grid frequency (ENF) component;Extract the average value feature of the change of ENF phase and frequency f hil As shallow feature;Determine frame number and frame length by the longest duration audio to be detected, and frame processing is carried out to ENF phase and f hil Phase feature matrix and frequency feature matrix are acquired;Curve fitting is carried out to obtain fitting coefficient;Local detail information of ENF is acquired from feature matrix using neural network, and global information of grid frequency is acquired from fitting coefficient to compensate, thereby forming deep feature, and fitting classification is carried out after shallow and deep feature fusion.The present application can effectively improve the recognition performance of system, improve the model generalization ability, optimize the system structure, and improve the competitiveness of corresponding equipment source identification product.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of digital audio tamper detection, and particularly refers to a digital audio tamper passive detection method based on fusion of deep and shallow features of power grid frequency. BACKGROUND

[0002] With the rapid progress of digital audio technology, people can easily collect digital audio signals, but at the same time, they can easily edit and modify them using many audio processing software. If such intentional or unintentional tampered digital audio is applied to important occasions such as judicial evidence, it will likely cause some undesirable social problems, so the research on digital audio tamper detection has very important significance.

[0003] Digital audio tamper passive detection is a technology that analyzes and discriminates the authenticity and integrity of digital audio only by the characteristics of the audio itself without adding any information, which has practical significance for complex forensic environments. When the recording equipment is powered by the power grid, the recorded audio file contains the Electirc Network Frequency (ENF) signal. When the digital audio is tampered, this ENF signal will also change with the tampering operation, so there are two research ideas for passive detection of audio tampering using the uniqueness and stability of the ENF signal. The first is to compare the ENF signal extracted from the audio with the ENF database of the power supply department, which is difficult to implement and costly. The second is to extract some features of the ENF signal for consistency and regularity analysis. The current research method for audio tamper forensics using ENF signal is mainly to use traditional machine learning methods to classify the phase change, phase discontinuity, and instantaneous frequency mutation of the ENF signal, so as to achieve the purpose of tamper detection.

[0004] Among the existing digital audio detection methods, most of them set threshold limits for detection or use machine learning methods for classification. These methods often have too many experience components or are too specific to a certain tampering method and have insufficient recognition rate.

[0005] In recent years, with the improvement of machine learning algorithm performance and the increase in computer storage and computing power, deep neural networks (DNNs) have been applied to the field of audio tampering detection. Deep neural networks can better fit the features of audio tampering through deep nonlinear transformations, achieving automatic learning and detection with the advantage of high recognition rate. Therefore, this invention uses deep learning methods to train the deep and shallow features of power grid frequencies, fully learning the implicit information with relatively little computation. Then, the attention mechanism is used to automatically learn the weights of the information in the deep and shallow features, determining the information useful for digital audio tampering detection, reducing redundant information, and improving the performance of the digital audio tampering detection system. Summary of the Invention

[0006] The above-mentioned technical problems of the present invention are mainly solved by the following technical solutions:

[0007] A passive detection method for digital audio tampering based on the fusion of deep and shallow features of power grid frequency, characterized by including:

[0008] The audio data to be detected is processed to obtain the power grid frequency (ENF) component, and then based on the DFT... 1 Transformation of the ENF components yields the ENF phase. and And based on the Hilbert transform, the instantaneous frequency f of the grid frequency (ENF) component is extracted. hil ;

[0009] Extracting ENF phase and frequency f hil The average value of the change is used as the shallow feature F. 123 =[F1,F2,F3]; Determine the number of frames and frame length based on the longest audio duration to be detected, and adjust the ENF phase. and f hil Perform frame segmentation processing to obtain the phase feature matrix P n×n and frequency characteristic matrix F m×m Perform curve fitting to obtain the fitting coefficient P. coe ,F coe ;

[0010] The local details of ENF are obtained from the feature matrix using a neural network, and global information compensation of the power grid frequency is obtained from the fitting coefficients. The local information and global information compensation together constitute deep features. After fusing the deep and shallow features, the system is fitted and classified.

[0011] In the above-mentioned digital audio tampering passive detection method based on power grid frequency shallow layer feature fusion, the original speech signal is processed to obtain the power grid frequency (ENF) component, specifically including:

[0012] Down-sampling sets the signal re-sampling frequency to 1000HZ or 1200HZ;

[0013] A 10000-order linear zero-phase FIR filter is used for narrowband filtering, with the center frequency at the ENF standard, the bandwidth being 0.6HZ, the passband ripple being 0.5dB, and the stopband attenuation being 100dB.

[0014] In the above-mentioned digital audio tampering passive detection method based on power grid frequency shallow layer feature fusion, the ENF phase includes:

[0015] Step 2.1, calculating the ENF signal X ENFC [n] is the approximate first-order derivative at point n

[0016] X′ ENFC [n]=f d (X ENFC [n]-X ENFC [n-1]) (1)

[0017] Where f d (*) represents the approximate derivative operation, X ENFC [n] represents the value of the nth point of the ENF component;

[0018] Step 2.2, frame and window X ENFC [n] and X′ ENFC [n] with the frame length being 10 standard ENF frequency periods The frame shift is 1 standard ENF frequency period X ENFC [n] and X′ ENFC [n] are windowed with the Hanning window w(n)

[0019] X N [n]=X ENF [n]w(n) (2)

[0020] X′ N [n]=X′ ENFC [n]w(n) (3)

[0021] Where the Hanning window L is the window length;

[0022] Step 2.3, each frame of signal X N [n] and X′ N[n] Perform N-point Discrete Fourier Transform (DFT) respectively to obtain X(k) and X′(k);

[0023] Step 2.4, let k peak The index of the peak value of |X(k)|; k peak Used for solving

[0024] Step 2.5: Estimate the frequency f from the ENF signal. DFT The phase characteristics of ENF can be determined.

[0025]

[0026] Step 2.6, Re-estimate DFT 1 Transformed ENF phase Let k be re-established peak This is the index of the peak value of |X′(k)|; and |X′(k)| is multiplied by a scaling factor F(k).

[0027]

[0028] Obtain DFT 0 [k] = X(k), DFT 1 [k] = F(k)|X′(k)|; therefore, the estimated frequency value is

[0029]

[0030] Step 2.7, k peak It should be the closest integers (f) d (for resampling frequency), so That would be a reasonable frequency value; it can be... Represented as

[0031]

[0032] in The value of θ is obtained by linear interpolation of X′(k), let... floor[a] represents the largest integer less than a, and ceil[b] represents the smallest integer greater than b;

[0033] because Therefore, in (k) low ,θ low )=arg[X′(k low )]and

[0034] (k high ,θ high )=arg[X′(khigh Linear interpolation can approximate the point The value obtained is consistent with the value of θ in the above formula;

[0035]

[0036] Step 2.8, the value obtained by the above method Has two possible values, so use As a reference, select The value closest to As the final

[0037] Step 2.9, extract the ENF instantaneous frequency f based on Hilbert transform hil , discrete Hilbert transform is carried out on the filtered ENF component; solve X ENF The analytic function of [n]

[0038] x ENFC [n] = X ENFC [n] + i * H{X ENFC [n]} (9)

[0039] Where H represents Hilbert transform; the instantaneous frequency f[n] is the rate of change of the phase angle of H{X ENFC [n]};

[0040] Step 2.10, estimate the loss frequency f[n] of the ENF component; the instantaneous frequency f[n] is the rate of change of the phase angle of H{X ENFC [n]};

[0041] Step 2.11, low-pass filter the obtained f[n] with a fifth-order elliptic IIR filter to remove the oscillation caused by numerical approximation in the Hilebrt transform process; the filter center frequency is the ENF nominal frequency, the bandwidth is 20HZ, and the passband ripple and stopband attenuation are 0.5HZ and 64HZ;

[0042] Step 2.12, remove the first and last 2000 sampling points of f[n]; reduce the influence of boundary effect on frequency estimation;

[0043] Step 2.13, get the final instantaneous frequency estimate f hil .

[0044] In the above method for passively detecting digital audio tampering based on the fusion of power grid frequency shallow and deep features, in step 3, the specific method for calculating the shallow feature includes:

[0045] Step 3.1, the phase with instantaneous frequency f hil Substitute into the following formula to obtain the statistical characteristics F reflecting the phase of ENF and the sudden change of instantaneous frequency 123 = [F1, F2, F3];

[0046]

[0047] wherein 2≤n b ≤N Block , is the estimated phase of the corresponding nth b frame, represents the average value from n b = 2 to N Block ; len = length(X ENFC [n]), f'(n) = f(n) - f(n-1), f(n) is the instantaneous frequency of the nth sampling point, m f′ represents the average value of f'(n) from n = 2 to len.

[0048] In the above-mentioned digital audio tampering passive detection method based on the fusion of power grid frequency shallow and deep features, in step 4, the specific method for obtaining the phase feature matrix P n×n , the frequency feature matrix F m×m , and the fitting coefficients P coe , F coe includes:

[0049] Step 4.1, obtaining the longest duration audio data in the audio data to be detected;

[0050] Step 4.2, performing DFT and Hilbert transform on the longest duration audio to obtain the phase and the instantaneous frequency f hil ;

[0051] Step 4.3, calculating the longest phase and frequency length len(f hil );

[0052] Step 4.4, calculating the frame length (m and n), wherein len(f hil ), wherein m is the frame length of the frequency feature matrix, and n is the frame length of the phase feature matrix;

[0053] Step 4.5, calculating the phase and the instantaneous frequency f hil of all audio data;

[0054] Step 4.6, calculating the frame shift and framing; the frame shift is

[0055]

[0056] Step 4.7: Reshape the phase and frequency after frame segmentation to obtain the feature matrix F. m×m P n×n ;

[0057] Step 4.8: Use the Sum of Sines function in the Fitting Toolbox of MATLAB to fit the phase and frequency. There are 6 Sum of Sines functions, yielding the characteristic curve P of the phase-frequency fitting coefficients. coe ,F coe =[a1,b1,c1,…,a i ,b i ,c i (0 < i ≤ 6); The Sum of Sines function is as follows:

[0058]

[0059] In the aforementioned passive detection method for digital audio tampering based on the fusion of deep and shallow features of power grid frequency, step 5 includes the following network model:

[0060] Step 5.1: Using the phase characteristic matrix P n×n Obtain detailed phase features; use two convolutional blocks to refine the phase feature matrix P. n×n Feature extraction is performed, and each convolutional block consists of two identical convolutional layers and one pooling layer (the number of convolutional kernels in the two convolutional blocks is 32 and 64 respectively; the kernel size is 3×3 and the stride is 1; the poolsize of the Maxpooling layer is 3); the last pooling layer outputs phase detail information and performs a Flatten operation.

[0061] Step 5.2: Using the phase fitting coefficient P coe Global information on deep phase features is obtained; phase fitting coefficients are fitted using two fully connected layers, with 32 neurons in each layer and ReLU activation function.

[0062] Step 5.3: Concatenate the global and local phase information to obtain deep phase features;

[0063] Step 5.4: Using the frequency feature matrix F m×m Obtain detailed information on deep frequency features; use three convolutional blocks to refine the phase feature matrix F. m×mFeature extraction is performed, and each convolutional block is composed of two identical convolutional layers and a pooling layer (the number of convolutional kernels of the three convolutional blocks is 32, 64, and 128; the size of the convolutional kernel is 3*3, and the step is 1; the poolsize of the Maxpooling layer is 3); the last pooling layer outputs phase detail information, and performs a Flatten operation;

[0064] Step 5.5, fitting coefficient F by frequency coe Global information of frequency deep features is obtained; phase fitting coefficients are fitted by two fully connected layers, and the number of neurons in each layer is 32, and the activation function is Relu;

[0065] Step 5.6, Concat of frequency global information and local information to obtain frequency deep features;

[0066] Step 5.7, using an attention mechanism to fuse phase deep features, frequency deep features, and shallow features; deep phase features, deep frequency features, and shallow features are spliced to obtain a feature quantity with a length of L;

[0067] Step 5.8, inputting the feature quantity with a length of L into two fully connected layers, and the activation functions are Relu and Sigmoid, respectively; the Relu activation function can enhance the nonlinearity of the network, and the weight of each feature is obtained through the Sigmoid activation function; finally, the obtained weight is multiplied by the spliced feature quantity with a length of L to realize feature selection;

[0068] Step 5.9, fitting and classifying the fused features; two fully connected layers are used to fully fit the features (the number of neurons is 1024 and 256, respectively, and the activation function is Relu); a Dropout layer (Dropout rate=0.2) is added between the two fully connected layers to prevent overfitting; finally, a fully connected layer (the number of neurons is 2, and the activation function is Softmax) is used as an output layer;

[0069] Step 5.10, the probability obtained by the output layer can determine whether the to-be-tested voice is tampered with, and the probability of correctly identifying whether all test voices are tampered with is calculated, that is, the recognition rate of the system.

[0070] A digital audio tampering passive detection device based on power grid frequency deep and shallow feature fusion, characterized in that it comprises

[0071] The first module: processing the to-be-tested audio data to obtain an electric network frequency (ENF) component, and based on DFT 1 transforming the ENF component to obtain an ENF phase and And based on the Hilbert transform extracts the instantaneous frequency f of the ENF component hil ;

[0072] Second module: extract the ENF phase And the average value feature of the change of frequency f hil As a shallow feature F 123 =[F1,F2,F3], and the ENF phase And f hil Frame processing to obtain the phase feature matrix P n×n And the frequency feature matrix F m×m ; Curve fitting to obtain the fitting coefficient P coe ,F coe ;

[0073] Third module: use neural network to obtain the local detailed information of ENF from the feature matrix, and obtain the global information compensation of power grid frequency from the fitting coefficient, and the local information and global information compensation jointly constitute the deep feature, and the deep and shallow features are fused and fitted for classification.

[0074] Therefore, the present application has the following advantages: compared with the traditional digital audio tampering detection, the present application proposes to use deep learning method and attention mechanism Attention to classify the deep and shallow features of ENF. In view of the problems of insufficient feature expression, much information loss and weak model generalization ability of the traditional method, the local detailed information of ENF is obtained from the feature matrix by using convolutional neural network (CNN), the global information compensation of power grid frequency is obtained from the fitting coefficient by using deep neural network (DNN), and the local information and global information compensation jointly constitute the deep feature. The attention mechanism screens out important information from the deep and shallow features, and reduces the operation burden of input data. Compared with the traditional digital audio tampering detection method, the digital audio tampering detection method of the present application can effectively improve the recognition performance of the system, improve the model generalization ability, optimize the system structure, and improve the competitiveness of the corresponding equipment source identification product. BRIEF DESCRIPTION OF DRAWINGS

[0075] Figure 1 is a method flowchart of the present application.

[0076] Figure 2 is a neural network structure diagram. DETAILED DESCRIPTION

[0077] The technical solutions of the present application will be further specifically described below by examples in combination with the drawings.

[0078] Example:

[0079] The application discloses a digital audio tampering passive detection method based on a convolutional neural network. Figure 1 As shown in a flowchart of an algorithm of the application, the method can be divided into four parts: 1) ENF component acquisition; 2) ENF phase and frequency feature extraction; 3) shallow feature acquisition; 4) input feature matrix and fitting coefficient acquisition of deep features; and 5) neural network training.

[0080] Step one: ENF component acquisition, the steps are as follows:

[0081] A, down-sampling the audio, and setting a re-sampling frequency to 1000HZ or 1200HZ;

[0082] B, using a 10000-order linear zero-phase FIR filter to perform narrowband filtering, a center frequency is at an ENF standard (50HZ or 60HZ), a bandwidth is 0.6HZ, a passband ripple is 0.5dB, and a stopband attenuation is 100dB;

[0083] Step two: ENF phase and frequency feature extraction, the steps are as follows:

[0084] A, calculating a first-order derivative of the signal, frame segmentation and windowing, discrete Fourier transform, linear interpolation estimation of the phase, and calculation of phase fluctuation features:

[0085] (A-1) calculating an ENF signal X ENFC [n] an approximate first-order derivative at point n

[0086] X′ ENFC [n]=f d (X ENFC [n]-X ENFC [n-1]) (1)

[0087] Wherein f d (*) represents an approximate derivation operation, and X ENFC [n] represents a value of the ENF component at the n th point.

[0088] (A-2) frame segmentation and windowing are performed on X ENFC [n] and X′ ENFC [n], a frame length is 10 standard ENF frequency periods A frame shift is 1 standard ENF frequency period X ENFC [n] and X′ ENFC [n] are windowed by using a Hanning window w(n)

[0089] X N [n]=X ENF [n]w(n) (2)

[0090] X′ N [n]=X′ENFC [n]w(n) (3)

[0091] where Hann window L is the window length.

[0092] (A-3) Let X N [n] and X' N [n] perform N-point discrete Fourier transform (DFT) to get X(k), X'(k), respectively.

[0093] (A-4) Let k peak be the index of the peak of |X(k)|. Let k peak be the index of the peak of |X'(k)|.

[0094] (A-5) From the estimated frequency f DFT of the ENF signal, the ENF phase feature

[0095]

[0096] (A-6) Re-estimate the DFT 1 transformed ENF phase Let k peak be the index of the peak of |X'(k)|. Multiply |X'(k)| by a scaling factor F(k)

[0097]

[0098] Get DFT 0 [k] = X(k), DFT 1 [k] = F(k)|X'(k)|. Thus the estimated frequency value is

[0099]

[0100] (A-7) k peak should be the integer closest to (f d is the resampling frequency), so that is a reasonable frequency value. F(k) may be expressed as

[0101]

[0102] where The value of θ is obtained by linear interpolation from X'(k), let floor[a] denotes the largest integer less than a, ceil[b] denotes the smallest integer greater than b.

[0103] Since Therefore, in (k) low ,θ low )=arg[X′(k low )] and (k high ,θ high )=arg[X′(k high Linear interpolation can approximate the point. The calculated value is consistent with the value of θ in the above formula.

[0104]

[0105] (A-8) The result obtained using the above method It has two possible values, therefore use For reference, choose closest The value is used as the final

[0106] B. Hilbert transform, low-pass filtering, instantaneous frequency estimation of ENF component extraction f hil :

[0107] (B-1) Extracting the instantaneous frequency f of ENF based on Hilbert transform hil The filtered ENF components are subjected to a discrete Hilbert transform. X is then solved. ENF analytic function of [n]

[0108] x ENFC [n] = X ENFC [n]+i*H{X ENFC [n]} (9)

[0109] in H represents the Hilbert transform. The instantaneous frequency f[n] is H{X} ENFC [n]} Rate of change of phase angle.

[0110] (B-2) Estimate the loss frequency f[n] of the ENF component. The instantaneous frequency f[n] is H{X ENFC [n]} Rate of change of phase angle.

[0111] (B-3) The obtained f[n] is low-pass filtered using a fifth-order elliptic IIR filter to remove oscillations caused by numerical approximation during the Hilebrt transform process. The filter's center frequency is the ENF nominal frequency, the bandwidth is 20 Hz, and the passband ripple and stopband attenuation are 0.5 Hz and 64 Hz, respectively.

[0112] (B-4) Remove 2000 sampling points from the beginning and end of f[n]. This reduces the impact of boundary effects in frequency estimation.

[0113] (B-5) get the final instantaneous frequency estimate f hil .

[0114] Step three: shallow feature acquisition, the steps are as follows:

[0115] A, the phase estimated above hil and the instantaneous frequency f 123 are substituted into the following formula to obtain the statistical characteristics F b reflecting the phase and instantaneous frequency of ENF mutation Block = [F1, F2, F3].

[0116]

[0117] where 2≤n b ≤N Block , is the estimated phase of the corresponding nth b frame, represents the average from n b = 2 to N Block . len = length (X ENFC [n]), f'(n) = f(n) - f(n-1), f(n) is the instantaneous frequency of the nth sampling point, m f′ represents the average of f'(n) from n = 2 to len.

[0118] Step four: input feature matrix and fitting coefficient acquisition of deep features.

[0119] A, the phase feature matrix P n×n and the frequency feature matrix F m×m are acquired.

[0120] (A-1) Acquire the longest duration audio data in the audio data to be detected.

[0121] (A-2) For the longest duration audio, DFT and Hilbert transform are used to acquire the phase and the instantaneous frequency f hil .

[0122] (A-3) Calculate the longest phase and frequency length len(f hil ).

[0123] (A-4) Calculate the frame length (m and n), where len(f hil ), where m is the frame length of the frequency feature matrix, and n is the frame length of the phase feature matrix;

[0124] (A-5) Calculate the phase of all audio data and instantaneous frequency f hil .

[0125] (A-6) Calculate the frame shift and divide the frame. The frame shift is...

[0126]

[0127] (A-7) Reshape the phase and frequency after framing to obtain the feature matrix F. m×m P n×n .

[0128] B. Fit coefficient P coe ,F coe Obtain.

[0129] The phase and frequency were fitted using the Sum of Sines function in MATLAB's Fitting Toolbox. The Sum of Sines function has six values, yielding the characteristic curve P of the phase-frequency fitting coefficients. coe ,F coe =[a1,b1,c1,…,a i ,b i ,c i (0 < i ≤ 6). The Sum of Sines function is as follows:

[0130]

[0131] Step 5: Network Model, the steps are as follows:

[0132] A. Through deep phase features.

[0133] (A-1) Through the phase characteristic matrix P n×n To obtain detailed phase features, two convolutional blocks are used to refine the phase feature matrix P. n×n Feature extraction is performed, with each convolutional block consisting of two identical convolutional layers and one pooling layer (the two convolutional blocks have 32 and 64 kernels respectively. The kernel size is 3×3, and the stride is 1. The Maxpooling layer has a poolsize of 3). The last pooling layer outputs phase detail information and performs a Flatten operation.

[0134] (A-2) Through the phase fitting coefficient P coe Global information on deep phase features is obtained. Phase fitting coefficients are fitted using two fully connected layers, with 32 neurons in each layer and ReLU activation function.

[0135] (A-3) Concat the global and local phase information to obtain deep phase features.

[0136] B, by frequency deep features.

[0137] (B-1) by frequency feature matrix F m×m Get frequency deep feature details. Use three convolutional blocks to phase feature matrix F m×m Feature extraction is performed, and each convolutional block is composed of two identical convolutional layers and a pooling layer (the number of convolutional kernels of the three convolutional blocks is 32, 64, and 128. The convolutional kernel size is 3x3, and the step is 1. The Maxpooling layer poolsize is 3). The last pooling layer outputs the phase detail information, and performs the Flatten operation.

[0138] (B-2) by frequency fitting coefficient F coe Get frequency deep feature global information. Use two fully connected layers to fit the phase fitting coefficient, and the number of neurons in each layer is 32, and the activation function is Relu.

[0139] (B-3) Concat the frequency global information and local information to get the frequency deep feature.

[0140] C, adopt attention mechanism to fuse deep and shallow features.

[0141] (C-1) Use attention mechanism to fuse phase deep features, frequency deep features and shallow features. The deep phase features, deep frequency features and shallow features are spliced to obtain a feature quantity with a length of L.

[0142] (C-2) Input the length L feature quantity into two fully connected layers, and the activation functions are Relu and Sigmoid respectively. The Relu activation function can enhance the nonlinearity of the network, and the weight of each feature is obtained through the Sigmoid activation function. Finally, the obtained weight is multiplied with the spliced length L feature quantity to realize feature selection.

[0143] D, decision classification.

[0144] (D-1) Fit and classify the features after feature fusion. Two fully connected layers are used to fully fit the features (the number of neurons is 1024 and 256 respectively, and the activation function is Relu). A Dropout layer (Dropout rate = 0.2) is added between the two fully connected layers to prevent overfitting. Finally, the fully connected layer (the number of neurons is 2, and the activation function is Softmax) is used as the output layer.

[0145] (D-2) The probability obtained by the output layer can determine whether the test voice is tampered with, and the probability of correctly identifying whether all test voices are tampered with is calculated, that is, the recognition rate of the system.

[0146] The specific embodiments described herein are merely illustrative of the principles of this application. Numerous modifications or adaptations will be readily apparent to those skilled in the art of the present application without departing from the spirit or scope of the present application as defined in the following claims.

Claims

1. A digital audio tampering passive detection method based on grid frequency deep and shallow feature fusion, characterized in that, Comprising The power grid frequency ENF component is obtained by processing the to-be-detected audio data, and the power grid frequency ENF component is extracted based on The ENF phase is obtained by processing the ENF component through transformation And The instantaneous frequency of the power grid frequency ENF component is extracted based on the Hilbert transformation ; extracting the average value feature of the change of the phase of the ENF , and frequency as a shallow feature ; determine the frame number and frame length by the longest duration audio to be detected, and perform frame processing on the phase of the ENF and to obtain a phase feature matrix and a frequency feature matrix ; perform curve fitting to obtain a fitting coefficient ; The local detailed information of ENF is obtained from the feature matrix by using a neural network, the global information of power grid frequency is obtained from the fitting coefficient for compensation, the local information and the global information compensation jointly constitute deep features, and the deep and shallow features are fused and fitted for classification. The network model part comprises: By phase feature matrix Obtain phase deep feature detail information; use two convolution blocks to process the phase feature matrix Extract features, each convolution block is composed of two identical convolution layers and a pooling layer, the number of convolution kernels of the two convolution blocks is 32, 64, the size of the convolution kernel is 3x3, the step is 1, and the poolsize of the Maxpooling layer is 3; the last pooling layer outputs the phase detail information and performs a Flatten operation; By phase fitting coefficients Obtain the global information of the phase deep features; use two layers of fully connected layers to fit the phase fitting coefficients, with 32 neurons in each layer and the activation function being Relu; The phase global information and the local information are concatenated to obtain phase deep features; By frequency feature matrix Obtain frequency deep feature detail information; use three convolution blocks to process the phase feature matrix Feature extraction is performed, each convolution block is composed of two same convolution layers and a pooling layer, the number of convolution kernels of the three convolution blocks is 32, 64 and 128, the size of the convolution kernel is 3*3, the step is 1, and the poolsize of the Maxpooling layer is 3; the last pooling layer outputs the phase detail information and performs a Flatten operation; By frequency fitting coefficients Obtain frequency deep feature global information; use two layers of full connection layer to fit phase fitting coefficients, and the number of neurons in each layer is 32, and the activation function is Relu; The frequency global information and the local information are concatenated to obtain frequency deep features; The phase deep feature, the frequency deep feature and the shallow feature are fused by using an attention mechanism; the deep phase feature, the deep frequency feature and the shallow feature are spliced to obtain a feature quantity with a length of ​ The length of the feature quantity is input into two full connection layers, and the activation functions thereof are Relu and Sigmoid respectively, the Relu activation function enhances the nonlinearity of the network, and the weight of each feature is obtained through the Sigmoid activation function; finally, the obtained weight is multiplied by the spliced feature quantity with the length of , so as to realize feature selection; The fused features are fitted and classified; two fully connected layers are used for fitting the features, the number of neurons of the two fully connected layers is 1024 and 256 respectively, and the activation function is Relu; a Dropout layer is added between the two fully connected layers, and the Dropout rate is 0.2 to prevent overfitting; finally, the fully connected layer is used as an output layer, the number of neurons of the fully connected layer is 2, and the activation function is Softmax; The probability obtained by the output layer can determine whether the to-be-tested voice is tampered with, and the probability of correctly identifying whether all test voices are tampered with is calculated, that is, the recognition rate of the system.

2. The digital audio tamper-detection method based on grid frequency deep and shallow feature fusion according to claim 1, characterized in that, The original voice signal is processed to obtain the power grid frequency ENF component, which specifically comprises: Downsampling sets the signal resampling frequency to 1000HZ or 1200HZ; A 10000-order linear zero-phase FIR filter is used for narrowband filtering, the center frequency is at the ENF standard, the bandwidth is 0.6HZ, the passband ripple is 0.5dB, and the stopband attenuation is 100dB.

3. The digital audio tamper-detection method based on grid frequency deep and shallow feature fusion according to claim 1, characterized in that, The ENF phase comprises: Step 2.1, calculating the ENF signal Approximate first derivative at point n (1) wherein denotes an approximate derivation operation, denotes the value of the ENF component at the point. Step 2.2, frame and window and frame and window with a Hanning window frame length of 10 standard ENF frequency periods frame shift of 1 standard ENF frequency period frame and window and frame and window with a Hanning window (2) (3) where the Hanning window , is the window length; Step 2.3, per frame signal and performing an N-point discrete Fourier transform, DFT, respectively, to obtain , ; Step 2.4, let be the index of the peak of ; for solving Step 2.5, obtaining the estimated frequency of the ENF signal , obtaining the ENF phase signature (4) Step 2.6, re-estimation Transformed ENF phase , re-assign to the index of the peak; and multiply by a scale factor (5) obtained , ; the estimated frequency value is therefore (6) Step 2.7、 should be the closest integer to the resampling frequency, so that is a reasonable frequency value; and is expressed as (7) wherein ; for values of , the values are linearly interpolated by , letting , , denotes the largest integer less than a, denotes the smallest integer greater than b; Since Therefore, in And linear interpolation approximation point , the value is consistent with the value of in the formula; (8) Step 2.8, the result is It has two possible values, therefore use For reference, choose closest The value is used as the final ; Step 2.9, Extracting the Hilbert Transform-based ENF Instantaneous Frequency performing a discrete Hilbert transform on the filtered ENF component; solving the analytic function (9) wherein H represents a Hilbert transform; instantaneous frequency is the rate of change of phase angle; Step 2.

10. Estimating the loss frequency of the ENF component ; instantaneous frequency is rate of change of phase angle; Step 2.

11. Low pass filtering the resulting with a 5th order elliptic IIR filter to remove ringing due to numerical approximation in the Hilbert transform; the filter center frequency is the nominal ENF frequency, the bandwidth is 20 HZ, the passband ripple and stopband attenuation are 0.5 HZ and 64 HZ; Step 2.12, removal 2000 samples at the head and tail; reduce the effect of boundary effects on frequency estimation; Step 2.13, obtain final instantaneous frequency estimate .

4. The digital audio tamper-detection method based on grid frequency deep and shallow feature fusion according to claim 1, characterized in that, The specific method for calculating the shallow features comprises: Step 3.1, estimate the phase of the ENF signal , and the instantaneous frequency Substitute into the following equation to obtain statistical characteristics reflecting the phase and instantaneous frequency of the ENF signal ; (10) wherein , , is the estimated phase of the respective frame, denotes the average value from to ; , , is the instantaneous frequency of the sample point, denotes the average value from to .

5. The digital audio tamper-detection method based on grid frequency deep and shallow feature fusion according to claim 1, characterized in that, acquiring a phase feature matrix , a frequency feature matrix , a fitting coefficient The specific method comprises: Step 4.1, obtaining the longest duration audio data in the to-be-detected audio data; Step 4.2, on the longest duration audio, DFT and Hilbert transform to get phase and instantaneous frequency ; Step 4.3, calculate the longest phase and frequency length ; Step 4.4, compute frame length and , wherein wherein is the frequency feature matrix frame length, is the phase feature matrix frame length; Step 4.5, calculate the phase of all audio data and instantaneous frequency ; Step 4.6, calculating the frame shift and framing; the frame shift is (11) Step 4.7, Reshape the framed phase and frequency to get the feature matrix , ; Step 4.8, fitting the phase and frequency using the Sum of Sines function in the fit toolbox in MATLAB, the number of Sum of Sines functions is 6, the phase frequency fitting coefficient characteristics are obtained ; the Sum of Sines function is as follows (12)。 6. A digital audio tampering passive detection device based on grid frequency deep and shallow feature fusion, adopting the method of any one of claims 1 to 5, characterized in that, Comprising The first module: processing the to-be-detected audio data to obtain a power grid frequency ENF component, and based on the ENF component The ENF phase is obtained by processing the ENF component And The instantaneous frequency of the power grid frequency ENF component is extracted based on the Hilbert transform ; Module 2: Extracting ENF Phase , and frequency The average value of the change is used as a shallow feature. and the ENF phase and Perform frame segmentation processing to obtain the phase feature matrix and frequency characteristic matrix Perform curve fitting to obtain the fitting coefficients. ; The third module: the local detailed information of ENF is obtained from the feature matrix by using a neural network, the global information of power grid frequency is obtained from the fitting coefficient for compensation, the local information and the global information compensation jointly constitute deep features, and the deep and shallow features are fused and fitted for classification.

Images