A voice lie detection method, device, medium and equipment

Abstract

This invention belongs to the field of speech signal processing technology and provides a method, apparatus, medium, and device for speech lie detection. The speech lie detection method includes: acquiring a speech signal to be tested and preprocessing it; decomposing the preprocessed speech signal and filtering out effective information, then reconstructing the speech signal using this effective information; extracting Mel-spectral coefficients (MSCs) from the reconstructed speech signal and calculating their first and second differences; stacking the extracted MSCs and their first and second differences to obtain stacked speech features; extracting time-series-based deep speech features from the stacked speech features; performing time-recursive processing on the time-series-based deep speech features to obtain the probabilities representing lies and truth, and determining the lie detection result.

Description

Technical Field

[0001] This invention belongs to the field of speech signal processing technology, and in particular relates to a method, apparatus, medium and device for detecting speech lies. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Lying can cause significant harm to society in certain fields (such as criminal investigation), making accurate lie detection crucial for solving cases. There are many ways to detect lies. Methods based on physiological signals require physical contact with the subject, while methods based on facial or posture have specific requirements regarding image angles. However, voice-based methods offer high concealment, reduce subject stress, and do not have special requirements for the location of the audio recording device, making them more authentic.

[0004] Researchers at Columbia University analyzed the effectiveness of machine learning and human detection methods on their constructed voice lie dataset CSC (Columbia-SRI-ColoradoCorpus). The experiments showed that the accuracy of lie detection by humans alone was far lower than that of automated machine detection methods. CV Ford et al. found that people who intentionally deceive others often exhibit subtle changes in a series of behaviors, including voice pressure, tone, speech rate, and vocal organs, when they are lying.

[0005] However, existing research on speech deception detection mainly focuses on the feature and classifier levels, lacking attention to the processing methods of the speech signal itself and insufficient mining of the temporal characteristics of the speech signal, which leads to a decrease in the accuracy of speech lie detection. Summary of the Invention

[0006] To address the technical problems existing in the background art, the present invention provides a method, apparatus, medium, and device for detecting speech lies. It utilizes the EMD method to decompose the signal and removes useless signal components based on a threshold, thereby improving the quality of the lie speech signal, enhancing the quality of the spectral features, and further mining the temporal information in the speech signal to achieve a higher lie detection accuracy.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: The first aspect of the present invention provides a method for detecting voice lies.

[0008] A method for detecting voice lies, comprising: Acquire the speech signal to be tested and preprocess it; The preprocessed speech signal to be tested is decomposed, and effective information is filtered out from it. The speech signal is then reconstructed using this effective information. The audio features Mel-Cepstral Coefficients are extracted from the reconstructed speech signal, and their first-order and second-order differences are calculated. The extracted audio features Mel-Cepstral Coefficients and their first-order and second-order differences are then stacked to obtain stacked speech features. Deep speech features based on time series are extracted from stacked speech features. These time series deep speech features are then subjected to time recursive processing to obtain the probabilities representing lies and truths, thus determining the lie detection result.

[0009] As one implementation method, the preprocessing of the acquired speech signal to be tested includes: Resample the speech signal at a set sampling rate; The sampled speech signal is pre-emphasized to enhance the high-frequency components of the sampled speech signal.

[0010] As one implementation method, the preprocessed speech signal to be tested is decomposed using the EMD method.

[0011] As one implementation method, the process of filtering out valid information is as follows: Decompose the speech signal into several sub-signals; Calculate the correlation coefficients between these sub-signals and the signal before decomposition; Valid sub-signals are selected based on a screening threshold. If the corresponding correlation coefficient is greater than the screening threshold, the corresponding sub-signal is retained; otherwise, the corresponding sub-signal is discarded.

[0012] As one implementation method, the process of extracting Mel-spectral coefficients, an audio feature, from the reconstructed speech signal includes: The reconstructed speech signal is framed and windowed. Perform a Fourier transform on each frame of signal that has undergone framing and windowing processing, and convert it into a frequency domain representation. The frequency domain signal is passed through a Mel filter bank to smooth the spectrum and eliminate harmonics, highlighting the formants of the original speech. Calculate the logarithmic energy of the output of each filter bank; The logarithmic energy is subjected to a discrete cosine transform to obtain the Mel-frequency cepstral coefficients.

[0013] As one implementation method, time-series-based deep speech features are extracted from stacked speech features using a TCN-LSTM network. The time-series-based deep speech features are then subjected to time recursive processing to obtain the probabilities representing lies and truths.

[0014] A second aspect of the present invention provides a speech lie detection device.

[0015] A voice lie detection device, comprising: The speech preprocessing module is used to acquire the speech signal to be tested and preprocess it. The speech signal reconstruction module is used to decompose the preprocessed speech signal to be tested, filter out the effective information, and reconstruct the speech signal using the effective information. The speech feature stacking module is used to extract the audio feature Mel-Cepstral coefficients from the reconstructed speech signal, calculate their first-order and second-order differences, and then stack the extracted audio feature Mel-Cepstral coefficients and their first-order and second-order differences to obtain stacked speech features. The lie detection module is used to extract time-series-based deep speech features from stacked speech features, perform time-series-based deep speech features through time recursion processing to obtain the probabilities representing lies and truths, and determine the lie detection result.

[0016] As one implementation method, the preprocessing process for the acquired speech signal to be tested in the speech preprocessing module includes: Resample the speech signal at a set sampling rate; The sampled speech signal is pre-emphasized to enhance the high-frequency components of the sampled speech signal.

[0017] As one implementation method, in the speech signal reconstruction module, the preprocessed speech signal to be tested is decomposed using the EMD method.

[0018] In one implementation, the speech signal reconstruction module decomposes the speech signal into several sub-signals; Calculate the correlation coefficients between these sub-signals and the signal before decomposition; Valid sub-signals are selected based on a screening threshold. If the corresponding correlation coefficient is greater than the screening threshold, the corresponding sub-signal is retained; otherwise, the corresponding sub-signal is discarded.

[0019] As one implementation, in the speech feature stacking module, the process of extracting Mel-spectral coefficients (MSCs) of audio features from the reconstructed speech signal includes: The reconstructed speech signal is framed and windowed. Perform a Fourier transform on each frame of signal that has undergone framing and windowing processing, and convert it into a frequency domain representation. The frequency domain signal is passed through a Mel filter bank to smooth the spectrum and eliminate harmonics, highlighting the formants of the original speech. Calculate the logarithmic energy of the output of each filter bank; The logarithmic energy is subjected to a discrete cosine transform to obtain the Mel-frequency cepstral coefficients.

[0020] As one implementation, in the lie detection module, time-series-based deep speech features are extracted from stacked speech features using a TCN-LSTM network. The time-series-based deep speech features are then subjected to time recursive processing to obtain the probabilities representing lies and truths.

[0021] A third aspect of the present invention provides a computer-readable storage medium.

[0022] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the speech lie detection method described above.

[0023] A fourth aspect of the present invention provides an electronic lie detector.

[0024] An electronic lie detection device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the voice lie detection method described above.

[0025] Compared with the prior art, the beneficial effects of the present invention are: This invention utilizes the EMD method to decompose the signal and removes useless signal components based on a threshold, thereby improving the quality of the lie speech signal and thus enhancing the quality of the spectral features. At the same time, by combining TCN and LSTM networks, it further mines the temporal information in the speech signal and achieves a higher lie detection accuracy.

[0026] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0027] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0028] Figure 1 This is a flowchart of the voice lie detection method according to an embodiment of the present invention; Figure 2 This is a flowchart of the speech signal EMD decomposition and reconstruction process according to an embodiment of the present invention; Figure 3 This is a flowchart of Mel-Cepstral Feature Extraction according to an embodiment of the present invention; Figure 4 This is a basic structural diagram of the TCN-LSTM network according to an embodiment of the present invention. Detailed Implementation

[0029] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0030] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0031] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0032] Example 1 Reference Figure 1 A method for detecting voice lies is presented, which includes: Step 1: Acquire the speech signal to be tested and preprocess it.

[0033] The specific process of step 1 includes: Step 1a: Collect voice signals under specific circumstances using recording equipment; Step 1b: Resample the speech signal at an 8kHz sampling rate; Step 1c: Pre-emphasize the sampled signal to enhance the high-frequency components and reduce the influence of lip radiation. The pre-emphasis formula is as follows: In the formula: The sampling signal is the first One sampling point, The first signal after pre-emphasis One sampling point, The value ranges from 0.9 to 1; in this invention, it is 0.97.

[0034] Step 2: Decompose the preprocessed speech signal to be tested, filter out the effective information, and reconstruct the speech signal using the effective information.

[0035] It should be noted here that the methods for decomposing the preprocessed speech signal to be tested can include EMD, LMD, VMD, or wavelet transform.

[0036] The EMD method is an adaptive decomposition method. The following section uses the EMD method as an example to explain in detail the process of decomposing the preprocessed speech signal under test.

[0037] like Figure 2 As shown, the EMD method is used to decompose the processed signal, filter out the useful parts, and reconstruct the speech signal using these signals. The specific steps are as follows: Step 2a: Apply the EMD method to decompose the speech signal into... indivual The specific steps for EMD decomposition are as follows: Step 2a (1): Locate the preprocessed speech signal The maximum and minimum values; Step 2a (2): Fit the upper and lower envelopes using the maximum and minimum points respectively; Step 2a (3): Calculate the average envelope based on the upper and lower envelopes; Step 2a(4): Subtract the average envelope from the speech signal to obtain the result. ; Step 2a(5): Replace the average envelope with Repeat steps 2a(1) to 2a(3) until the mean envelope is monotonic or The number has reached the set value.

[0038] Step 2b: Calculate these respectively Correlation coefficient with the signal before decomposition The correlation coefficient is calculated using the Pearson correlation coefficient, and the specific formula is as follows: In the formula, N is the total number of speech sampling points. The first one obtained from EMD decomposition A sub-signal, The signal before decomposition, and This represents the mean of all sampling points of the two signals. The range of values ​​is within between.

[0039] Step 2c: According to Calculate the filtering threshold Filter effective If the corresponding correlation coefficient is greater than the threshold Then keep that For the component, this embodiment sets a threshold based on statistical methods, and the calculation formula is as follows: In the formula, The total number of decompositions. This is the mean of all correlation coefficients.

[0040] The threshold set by statistical methods is calculated based on the information of each speech signal itself. Compared with fixed thresholds, it is more targeted to each data point. Other methods, such as wavelet thresholds, require wavelet transform of the signal first. In comparison, this method is simple and intuitive to calculate and has good results.

[0041] It is understood here that in other embodiments, other existing methods may be used to set the threshold, which will not be described in detail here.

[0042] Step 2d: Will be effective The combined signals are then reconstructed to obtain the speech signal.

[0043] Step 3: Extract the audio feature Mel-Cepstral Coefficients from the reconstructed speech signal, calculate their first-order and second-order differences, and then stack the extracted audio feature Mel-Cepstral Coefficients and their first-order and second-order differences to obtain stacked speech features.

[0044] like Figure 3 As shown, the Mel-spectral coefficients, audio features, are extracted from the reconstructed speech signal. Their first and second differences are calculated, and all features are stacked to obtain the final input features. The specific steps are as follows: Step 3a: The reconstructed signal is framed and windowed to ensure the smoothness of the speech signal. In this invention, the length of each frame is set to 25ms, the frame shift is 15ms, and the window function is set to a Hamming window, the formula of which is as follows: In the formula, N represents the window length; Step 3b: Perform a Fourier transform on each frame of signal to convert it into a frequency domain representation. The discrete Fourier transform formula is as follows: In the formula, For the signal after framing and windowing, The length of each frame of signal; Step 3c: Pass the frequency domain represented signal through a Mel filter bank to smooth the spectrum and eliminate harmonics, highlighting the formants of the original speech. The frequency response of the Mel filter bank is defined as: In the formula, Indicates the first The frequency response of each filter Indicates the first The center frequency of each filter, and the sum of the frequency responses of all filters equals 1; Step 3d: Calculate the logarithmic energy of the output of each filter bank, using the following formula: In the formula, This represents the total number of filters; Step 3e: Perform a discrete cosine transform on the logarithmic energy to obtain the Mel-frequency cepstral coefficients. The formula for the discrete cosine transform is as follows: In the formula, The order of the obtained Mel-Cepstral coefficients is usually taken as the first 12-16th order, and the first 13th order is taken in this invention; Step 3f: Calculate the first and second difference features of the Mel-frequency cepstral coefficients. Stack the Mel-frequency cepstral coefficients and their first and second difference features to obtain the final 39-dimensional features. Stacking refers to concatenating 13*N MFCC coefficients, 13*N first-order differences of MFCC coefficients, and 13*N second-order differences of MFCC coefficients to form a 39*N feature. N represents the number of frames in the speech segmentation.

[0045] Step 4: Extract time-series-based deep speech features from the stacked speech features, perform time-recursive processing on the time-series-based deep speech features to obtain the probabilities representing lies and truths, and determine the lie detection result.

[0046] In this embodiment, time-series-based deep speech features are extracted from stacked speech features using a TCN-LSTM network. The time-series-based deep speech features are then subjected to time recursive processing to obtain the probabilities representing lies and truths.

[0047] The specific steps for constructing a TCN-LSTM network for lie detection and outputting the lie detection results are as follows: Step 4a: The basic structure of the TCN-LSTM network is as follows Figure 4 As shown, the specific parameter settings are as follows: the TCN layer has 4 layers, the number of channels in the three hidden layers is 64, 32, and 16 respectively, the convolution kernel size is 3, and the dilation coefficients are 1, 2, 4, and 8. The 39×N dimensional (N is the number of frames obtained from the segmentation of the speech signal) MFCC features are passed through the TCN network to obtain 16×N dimensional depth features. The LSTM layer has 1 layer, and the hidden layer dimension is set to 4. The LSTM receives the output of the TCN and processes it into a 4×N dimensional output. The TCN-LSTM is followed by two linear layers. A 0.3 dropout layer is used before the input linear layers to prevent overfitting. The output of the LSTM is flattened into a 4×N dimensional one-dimensional feature vector. After passing through the linear layers, the final output is a 2-dimensional prediction result, with the two values ​​representing the probabilities of lying and telling the truth, respectively. The activation function in the entire network is ReLU; the model optimizer is Adam, the learning rate is set to 0.001, the loss function is the cross-entropy function, the number of iterations is set to 100, and the batch size is set to 64. The formula for the cross-entropy loss function is as follows: In the formula Indicates the number of samples. Indicates the corresponding sample. Represents the true value of the sample. and These represent the predicted probabilities of the output being a lie and the output being the truth, respectively.

[0048] Step 4b: Compare the prediction probabilities of lies and truths; the one with the higher probability is the prediction result.

[0049] To verify the scientific validity and effectiveness of this invention, experiments were conducted using the publicly available Real-life Trail dataset. This dataset contains 121 video clips of court trials, of which 61 are deceptive and 60 are genuine. The speakers in the dataset include 21 women and 35 men, aged 16 to 60. This invention detects lies based on speech, using only the speech portion of the dataset. The dataset was segmented to obtain 1287 short clips, each 3 seconds long.

[0050] The computer used in the experiments of this invention was configured with: an Intel® Core™ i7-10700 CPU @2.9GHz processor, 16GB of memory, an NVIDIA GeForce RTX 3060 graphics card, and a Windows 10 operating system. The PyCharm compiler was used, and programs were written based on the PyTorch deep learning framework and the sklearn machine learning package.

[0051] First, the impact of different sampling rates on the present invention was analyzed and compared, and the results are shown in Table 1.

[0052] Table 1. Experimental results at different sampling rates As shown in Table 1, the results at a 4kHz sampling rate are not ideal. Although EMD calculation is faster at lower sampling rates, 4kHz is insufficient to retain enough information from the original speech, resulting in the loss of some useful information. The results at 8kHz and 16kHz show no significant difference, indicating that sufficient useful information can be retained at both sampling rates. Considering that EMD calculation is faster at 8kHz, and 16kHz does not show a significant advantage, 8kHz is used for speech resampling in this invention.

[0053] Table 1 also shows that the EMD decomposition and reconstruction method proposed in this invention significantly improves accuracy; similarly, the TCN-LSTM network used in this invention achieves better performance than using TCN or LSTM alone. This demonstrates that this invention has certain advantages in speech lie detection.

[0054] Secondly, to fully verify the superiority of the proposed scheme, some supplementary experiments were conducted. Firstly, regarding machine learning algorithms, the following classic algorithms were selected: KNN (K-Nearest Neighbors), SVM (Support Vector Machine), Decision Tree, Random Forest, Adaboost, and Ensemble Learning. Secondly, MLP (Multilayer Perceptron), autoencoders, and Bi-LSTM (Bidirectional Long Short-Term Memory) networks were used as supplementary comparisons of neural networks. The results are shown in Table 2.

[0055] Table 2 Experimental results under different classifiers Classification Algorithm Untreated by EMD EMD processing KNN 77.3% 78.8% SVM 77.5% 79.6% Decision Tree 70.0% 70.8% Random Forest 78.6% 79.6% Adaboost 72.8% 72.9% Ensemble Learning 79.2% 81.2% MLP 72.1% 76.6% Bi-LSTM 68.3% 70.2% Self-encoder 67.4% 72.9% TCN-LSTM 80.6% 85.6% According to the results in Table 2, the detection accuracy was improved after adopting the scheme of this invention. However, because neural networks are more sensitive to the speech features processed by EMD and can learn deeper feature information, the improvement of the EMD method in traditional machine learning algorithms is smaller than that of the neural network algorithm. Analyzing all the algorithms in Table 2, the highest accuracy of other methods is 81.2%, which is about 4% worse than the proposed TCN-LSTM.

[0056] Finally, in addition to the EMD-based method, similar signal decomposition methods LMD (Local Mean Decomposition) and VMD (Variational Mode Decomposition) were also used for comparison with EMD. The results are shown in Table 3.

[0057] Table 3 Comparison of different signal decomposition methods Signal decomposition methods result EMD 85.6% LMD 83.2% VMD 78.6% According to the results in Table 3, among the three methods, the EMD-based scheme performed best, while the VMD-based scheme performed worst, even worse than the detection performance of not processing the signal. Analyzing the VMD algorithm, it decomposes the original signal into a specified number of components by constructing and solving a variational problem. In short, due to the allowable error in the convergence condition of the constructed variational problem, even if all components are retained to reconstruct the signal, there will still be differences compared to the original signal. This difference affects speech lie detection. Applying LMD can improve the accuracy of lie detection, but the results in this experiment are slightly worse than EMD, warranting further investigation.

[0058] The three experiments on the Real-lifeTrail dataset strongly illustrate the role of this invention. Applying EMD to decompose and reconstruct the signal helps to obtain higher quality speech features. The combination of TCN and LSTM learns the temporal information in the speech signal more deeply, thereby improving the performance of speech lie detection.

[0059] Example 2 This embodiment provides a voice lie detection device, which includes: (1) Speech preprocessing module, which is used to acquire the speech signal to be tested and preprocess it.

[0060] Specifically, in the speech preprocessing module, the process of preprocessing the acquired speech signal to be tested includes: Resample the speech signal at a set sampling rate; The sampled speech signal is pre-emphasized to enhance the high-frequency components of the sampled speech signal.

[0061] (2) Speech signal reconstruction module, which is used to decompose the preprocessed speech signal to be tested, and to filter out the effective information from it and reconstruct the speech signal using the effective information.

[0062] Specifically, in the speech signal reconstruction module, the EMD method is used to decompose the preprocessed speech signal to be tested.

[0063] Specifically, in the speech signal reconstruction module, the speech signal is decomposed into several sub-signals; Calculate the correlation coefficients between these sub-signals and the signal before decomposition; Valid sub-signals are selected based on a screening threshold. If the corresponding correlation coefficient is greater than the screening threshold, the corresponding sub-signal is retained; otherwise, the corresponding sub-signal is discarded.

[0064] (3) Speech feature stacking module, which is used to extract the audio feature Mel-Cepstral coefficients from the reconstructed speech signal, calculate its first-order difference and second-order difference, and then stack the extracted audio feature Mel-Cepstral coefficients and its first-order difference and second-order difference to obtain stacked speech features.

[0065] Specifically, in the speech feature stacking module, the process of extracting Mel-spectral coefficients of audio features from the reconstructed speech signal includes: The reconstructed speech signal is framed and windowed. Perform a Fourier transform on each frame of signal that has undergone framing and windowing processing, and convert it into a frequency domain representation. The frequency domain signal is passed through a Mel filter bank to smooth the spectrum and eliminate harmonics, highlighting the formants of the original speech. Calculate the logarithmic energy of the output of each filter bank; The logarithmic energy is subjected to a discrete cosine transform to obtain the Mel-frequency cepstral coefficients.

[0066] (4) Lie detection module, which is used to extract time-series-based deep speech features from stacked speech features, perform time-series-based deep speech features through time recursion processing, obtain the probabilities representing lies and truths, and determine the lie detection results.

[0067] Specifically, in the lie detection module, time-series-based deep speech features are extracted from stacked speech features using a TCN-LSTM network. The time-series-based deep speech features are then subjected to time recursive processing to obtain the probabilities representing lies and truths.

[0068] It should be noted that each module in this embodiment corresponds one-to-one with each step in Embodiment 1, and their specific implementation processes are the same, so they will not be repeated here.

[0069] Example 3 This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the voice lie detection method described above.

[0070] Example 4 This embodiment provides an electronic lie detection device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the voice lie detection method described above.

[0071] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0072] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting voice lies, characterized in that, include: Acquire the speech signal to be tested and preprocess it; The preprocessed speech signal to be tested is decomposed, and effective information is filtered out from it. The speech signal is then reconstructed using this effective information. The audio features Mel-Cepstral Coefficients are extracted from the reconstructed speech signal, and their first-order and second-order differences are calculated. The extracted audio features Mel-Cepstral Coefficients and their first-order and second-order differences are then stacked to obtain stacked speech features. Based on the TCN-LSTM network, time-series-based deep speech features are extracted from stacked speech features. The time-series-based deep speech features are then processed recursively over time to obtain the probabilities representing lies and truths, thus determining the lie detection result. Constructing a TCN-LSTM network for lie detection and outputting the lie detection results, the specific steps are as follows: The specific parameter settings of the TCN-LSTM network are as follows: the TCN layer has 4 layers, the number of channels in the three hidden layers is 64, 32, and 16 respectively, the convolution kernel size is 3, and the dilation coefficients are 1, 2, 4, and 8. The 39×N dimensional MFCC features are passed through the TCN network to obtain 16×N dimensional depth features, where N is the number of frames obtained by segmenting the speech signal. The LSTM layer has 1 layer, and the hidden layer dimension is set to 4. The LSTM receives the output of the TCN and processes it into a 4×N dimensional output. The TCN-LSTM is followed by two linear layers. A 0.3 dropout layer is used before the input linear layers to prevent overfitting. The output of the LSTM is flattened into a 4×N dimensional one-dimensional feature vector. After passing through the linear layers, the final output is a 2-dimensional prediction result, with the two values ​​representing the probabilities of lying and telling the truth, respectively. The activation function in the entire network is ReLU. The model optimizer is Adam, the learning rate is set to 0.001, the loss function is the cross-entropy function, the number of iterations is set to 100, and the batch size is set to 64. The formula for the cross-entropy loss function is as follows: In the formula Indicates the number of samples. Indicates the corresponding sample. Represents the true value of the sample. and These represent the predicted probabilities of the output being a lie and the output being the truth, respectively.

2. The voice lie detection method as described in claim 1, characterized in that, The preprocessing of the acquired speech signal to be tested includes: Resample the speech signal at a set sampling rate; The sampled speech signal is pre-emphasized to enhance the high-frequency components of the sampled speech signal.

3. The voice lie detection method as described in claim 1, characterized in that, The preprocessed speech signal to be tested was decomposed using the EMD method.

4. The voice lie detection method as described in claim 1, characterized in that, The process of filtering out valid information is as follows: Decompose the speech signal into several sub-signals; Calculate the correlation coefficients between these sub-signals and the signal before decomposition; Valid sub-signals are selected based on a screening threshold. If the corresponding correlation coefficient is greater than the screening threshold, the corresponding sub-signal is retained; otherwise, the corresponding sub-signal is discarded.

5. The voice lie detection method as described in claim 1, characterized in that, The process of extracting Mel-spectral coefficients, an audio feature from a reconstructed speech signal, includes: The reconstructed speech signal is framed and windowed. Perform a Fourier transform on each frame of signal that has undergone framing and windowing processing, and convert it into a frequency domain representation. The frequency domain signal is passed through a Mel filter bank to smooth the spectrum and eliminate harmonics, highlighting the formants of the original speech. Calculate the logarithmic energy of the output of each filter bank; The logarithmic energy is subjected to a discrete cosine transform to obtain the Mel-frequency cepstral coefficients.

6. A voice lie detection device, characterized in that, include: The speech preprocessing module is used to acquire the speech signal to be tested and preprocess it. The speech signal reconstruction module is used to decompose the preprocessed speech signal to be tested, filter out the effective information, and reconstruct the speech signal using the effective information. The speech feature stacking module is used to extract the audio feature Mel-Cepstral coefficients from the reconstructed speech signal, calculate their first-order and second-order differences, and then stack the extracted audio feature Mel-Cepstral coefficients and their first-order and second-order differences to obtain stacked speech features. The lie detection module is used to extract time-series-based deep speech features from stacked speech features based on the TCN-LSTM network, perform time-series-based deep speech features through time recursion processing to obtain the probabilities representing lies and truths, and determine the lie detection result. Constructing a TCN-LSTM network for lie detection and outputting the lie detection results, the specific steps are as follows: The specific parameter settings of the TCN-LSTM network are as follows: the TCN layer has 4 layers, the number of channels in the three hidden layers is 64, 32, and 16 respectively, the convolution kernel size is 3, and the dilation coefficients are 1, 2, 4, and 8. The 39×N dimensional MFCC features are passed through the TCN network to obtain 16×N dimensional depth features, where N is the number of frames obtained by segmenting the speech signal. The LSTM layer has 1 layer, and the hidden layer dimension is set to 4. The LSTM receives the output of the TCN and processes it into a 4×N dimensional output. The TCN-LSTM is followed by two linear layers. A 0.3 dropout layer is used before the input linear layers to prevent overfitting. The output of the LSTM is flattened into a 4×N dimensional one-dimensional feature vector. After passing through the linear layers, the final output is a 2-dimensional prediction result, with the two values ​​representing the probabilities of lying and telling the truth, respectively. The activation function in the entire network is ReLU. The model optimizer is Adam, the learning rate is set to 0.001, the loss function is the cross-entropy function, the number of iterations is set to 100, and the batch size is set to 64. The formula for the cross-entropy loss function is as follows: In the formula Indicates the number of samples. Indicates the corresponding sample. Represents the true value of the sample. and These represent the predicted probabilities of the output being a lie and the output being the truth, respectively.

7. The voice lie detection device as described in claim 6, characterized in that, In the speech preprocessing module, the process of preprocessing the acquired speech signal to be tested includes: Resample the speech signal at a set sampling rate; The sampled speech signal is pre-emphasized to enhance the high-frequency components of the sampled speech signal; or In the speech signal reconstruction module, the EMD method is used to decompose the preprocessed speech signal to be tested; or In the speech signal reconstruction module, the speech signal is decomposed into several sub-signals; Calculate the correlation coefficients between these sub-signals and the signal before decomposition; Valid sub-signals are selected based on a screening threshold. If the corresponding correlation coefficient is greater than the screening threshold, the corresponding sub-signal is retained; otherwise, the corresponding sub-signal is discarded. or In the speech feature stacking module, the process of extracting Mel-spectral coefficients (MSCs) from the reconstructed speech signal includes: The reconstructed speech signal is framed and windowed. Perform a Fourier transform on each frame of signal that has undergone framing and windowing processing, and convert it into a frequency domain representation. The frequency domain signal is passed through a Mel filter bank to smooth the spectrum and eliminate harmonics, highlighting the formants of the original speech. Calculate the logarithmic energy of the output of each filter bank; The logarithmic energy is subjected to a discrete cosine transform to obtain the Mel-frequency cepstral coefficients.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the voice lie detection method as described in any one of claims 1-5.

9. An electronic lie detector, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the voice lie detection method as described in any one of claims 1-5.

Images