Video lying detection method based on capsule network structure search and uncertainty evaluation

Abstract

This invention discloses a video deception detection method based on capsule network structure search and uncertainty evaluation, comprising: 1. extracting heart rate features and facial features from video samples to form deception detection features; 2. constructing a channel attention layer, a pre-convolutional layer, a main capsule layer, and a digital capsule layer, and sampling to generate a network structure population; 3. constructing a boundary loss function and an uncertainty loss function; 4. training each individual network structure in the network structure population, using accuracy and uncertainty as evaluation indicators for the individual network structure to generate the next generation of the network structure population; 5. repeating the network search process until the maximum number of iterations is reached, thereby outputting the optimal deception detection network individual; 6. retraining the optimal deception detection network individual. This invention can detect deception based on deception detection features and evaluate the uncertainty of the detection results, providing a solution for contactless, automatic deception detection based on videos.

Description

Technical Field

[0001] The present invention belongs to the technical field of biomedical signal processing, and in particular relates to a method for video lie detection and uncertainty evaluation based on capsule network structure search. Background Art

[0002] Lying is becoming increasingly common in our daily interactions and can have serious consequences. Therefore, effective lie detection methods are crucial for maintaining social order. Currently, modalities such as video, audio, and text are widely used for lie detection. Video contains rich multimodal information, such as facial expressions, body movements, and sounds, making it very useful for lie detection. At the same time, video data is becoming increasingly accessible, and people can easily record and share video content through various devices (such as smartphones and cameras), providing a wide range of data sources and application scenarios for video lie detection. However, video lie detection is very challenging.

[0003] While deep learning techniques, particularly convolutional neural networks, have demonstrated significant advantages in lie detection, existing network architectures are mostly designed by human experts, requiring prior knowledge and experience, a time-consuming and error-prone process. Furthermore, due to inherent human thinking patterns, it is difficult for experts to design ideal models. Therefore, there is a desire to automate the search for network architectures through algorithms, which could significantly free up researchers' creativity and reduce the heavy cost of network design.

[0004] Because deception is rarely an outright lie, lie detection inherently involves uncertainty. When an expert assesses whether a person is lying, they may reach a conclusion, but rarely with 100% certainty. Therefore, reasoning about and making decisions about lying are inherently uncertain, making uncertainty measures crucial for lie detection.

[0005] Therefore, how to automatically design high-performance networks based on deep learning technology and evaluate the uncertainty of the network's classification results is an urgent problem that needs to be solved in video-based lie detection. Summary of the Invention

[0006] The present invention addresses the shortcomings of the existing technology and provides a video lie detection method based on capsule network structure search and uncertainty evaluation, aiming to reduce the uncertainty of lie detection results and improve the accuracy and credibility of lie detection.

[0007] The present invention adopts the following scheme to solve the technical problem:

[0008] The video deception detection method based on capsule network structure search and uncertainty evaluation of the present invention is characterized in that it is performed according to the following steps:

[0009] Step 1: Extract video samples After concatenating the visual features and heart rate features, we get the lie detection feature matrix , After fusion, The lying detection feature vector of each channel, Indicates the number of channels of the lie detection feature matrix;

[0010] Step 2: Construct a network space consisting of a channel attention layer, a pre-convolutional layer, a main capsule layer, and a digital capsule layer ; Wherein, the channel attention layer includes: an efficient channel attention mechanism module and a selective kernel network module, and the serial number of any module in the channel attention layer is recorded as The pre-convolution layer includes: a two-layer convolution module, a three-layer convolution module and a four-layer convolution module, and the serial number of any module in the pre-convolution layer is recorded as ; The main capsule layer comprises: The main capsule modules with different output dimensions are recorded as ; The digital capsule layer includes: digital capsule modules with different output dimensions, and the serial number of any module in the digital capsule layer is recorded as , ; ; ; ;

[0011] Step 3: Use uncertainty-based search algorithms to search from network space Search for the optimal network structure ;

[0012] Step 4: Enter the Optimal Lie Detection Network After processing and obtaining the optimal prediction category, the optimal total loss function is constructed according to formula (4) to be used for the optimal lie detection network Train to update The weight parameters are adjusted until the total number of training times is reached. So far, the optimal lie detection model after training is obtained to realize lie detection.

[0013] Furthermore, step 1 is performed as follows:

[0014] Step 1.1: For any subject A video sample of Perform frame-level analysis and extraction to obtain the subject The facial feature matrix includes: Eye gaze direction matrix for the frame , No. Head pose matrix for the frame , No. Facial action unit matrix for a frame ;in, Indicates the Frame No. The eye gaze direction vector of each channel, Indicates the Frame No. Channel head pose vector, Indicates the Frame No. channels of facial action unit vectors, , , , , express The number of channels, express The number of channels, express The number of channels; F represents the video sample The total number of frames;

[0015] For the video sample The corresponding category label is encoded to obtain the Category code value ;when When The category is lying, when hour, The category is not lying;

[0016] Step 1.2: From video sample Extract the heart rate signal vector and use the cubic spline interpolation method to adjust the data length of the heart rate signal vector to obtain the heart rate feature vector , Indicates the Heart rate feature value of the frame;

[0017] Step 1.3: According to the channel dimension Eye gaze direction matrix for the frame 、 Head pose matrix for the frame 、 Facial action unit matrix for a frame , heart rate feature vector Splice and get the lie detection feature matrix , After fusion, The lying detection feature vector of each channel, represents the number of channels of the lie detection feature matrix, and =M+N+O+1, .

[0018] Furthermore, step 3 is performed as follows:

[0019] Step 3.1, the current number of iterations is , the total number of iterations is , set the Generation network structure population The total number of individuals in the network structure is , the serial number of the current network individual is ,initialization ;

[0020] A module is randomly selected from the channel attention layer, pre-convolution layer, main capsule layer and digital capsule layer to form the first Generation network structure population The network structure individuals, thus constructing the Generation network structure population in A network structure individual;

[0021] Step 3.2, Initialization ;

[0022] Step 3.3, Enter Generation network structure population The In the individual network structures, the first Modules are processed to obtain Generation network structure population The Lie detection attention feature matrix ;

[0023] Will Enter Generation network structure population The The first one selected in the pre-convolutional layer of each network structure Modules are processed to obtain Generation network structure population The Lie detection pre-feature matrix ;

[0024] Will Enter Generation network structure population The The first one selected in the main capsule layer of the network structure individual Modules are processed to obtain Generation network structure population The lie detection main capsule feature matrix ;

[0025] Will Enter Generation network structure population The The first one selected in the digital capsule layer of the network structure individual Modules are processed to obtain Generation network structure population The Lie detection state capsule sequence , Indicates the Generation network structure population The The first one in the lie detection state capsule sequence Class state capsules, when hour, The state capsule representing the lying category, when hour, State capsule representing the true category;

[0026] Step 3.4, calculation The L2 norm of Predicted class probabilities of lying , and select the probability of lying prediction category represented by the larger L2 norm as the prediction category of video sample V; calculate the evidence of lying prediction category ,in, represents the rectified linear unit function;

[0027] Step 3.5, Build The The total loss function of each network structure ;

[0028] Step 3.6: Use the stochastic gradient descent algorithm to The Each network structure is trained individually, and the calculation To continuously adjust The weight parameters of the network structure network individuals until the maximum number of training times is reached So far, we can get the first a lie detection network individual;

[0029] Step 3.7, Input the training The predicted categories are used to calculate the first Daidi The accuracy rate is used as the The first evaluation index of a lie detection network individual , use formula (5) to calculate the Daidi uncertainty , and As the first The second evaluation index of the lie detection network individuals ;

[0030] (5)

[0031] Step 3.8, let Assign to After that, judge Is it established? If it is established, it means that the After generation training lie detection network individuals and constitute the first Generation of lie detection network populations and the two evaluation indicators of each lie detection network individual, and execute step 3.9; otherwise, return to step 3.3 and execute sequentially;

[0032] Step 3.9, according to the trained Two evaluation indicators for each individual in the lie detection network Each lie detection network individual performs non-dominated sorting and calculates the crowding distance, thereby selecting the one with the largest crowding distance on the Pareto front. A lie detection network individual, and as the first Generation population in A network structure individual;

[0033] Step 3.10, randomly select one module from the channel attention layer, pre-convolution layer, main capsule layer and digital capsule layer, and form the same Generation population A network structure individual with different individual structures, thus obtaining the first Generation network structure population In addition A network structure individual;

[0034] Step 3.11, let Assign to After that, judge Is it true? If so, it means that the T-th generation population is obtained. , and execute step 4, otherwise, return to step 3.1 and execute sequentially;

[0035] Step 3.12: Based on the T generation population Two evaluation indicators for each individual lie detection network in Perform non-dominated sorting on each lie detection network individual in the network, and select the lie detection network individual with the smallest index on the Pareto front as the optimal lie detection network .

[0036] Furthermore, step 3.5 is performed as follows:

[0037] Step 3.5.1: Use formula (1) to construct The The first Boundary loss function for each category :

[0038] (1)

[0039] In formula (1), yes If the category prediction is correct, then =1, otherwise, let =0; The hyperparameter representing the minimum distance between the correct and incorrect predicted categories, A hyperparameter representing the maximum distance between the correct and incorrect predicted classes; The weight that controls the degree of penalty for incorrectly predicted categories;

[0040] Step 3.5.2: Use formula (2) to construct The The sum of the boundary loss functions of individual network structures ;

[0041] (2)

[0042] Step 3.5.3: Use formula (3) to construct The Uncertainty loss function of individual network structures :

[0043] (3)

[0044] In formula (3), It is Daidi The intensity parameter of the Dirichlet distribution of the individual network structures, and , is the annealing coefficient, yes Divergence, represents the Dirichlet distribution, express Middle The probability matrix of the lying prediction category assignment of the individual network structure is , represents the Dirichlet distribution parameter whose value is 1, express The The Dirichlet distribution parameter of the individual network structure is , ; express The The first Dirichlet distribution parameter for each category;

[0045] Step 3.5.4: Use formula (4) to construct The The total loss function of each network structure ;

[0046] (4).

[0047] The electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the video lie detection method, and the processor is configured to execute the program stored in the memory.

[0048] The present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program executes the steps of the video lying detection method when executed by a processor.

[0049] Compared with the prior art, the beneficial effects of the present invention are embodied in:

[0050] 1. The present invention utilizes neural network architecture search technology to adaptively optimize the structure and parameters of the lie detection network, making the lie detection network model perform better, thereby significantly improving the accuracy of lie detection.

[0051] 2. This invention also establishes four search space modules, including a channel attention module, a pre-convolution module, a main capsule module, and a digital capsule module. This diverse design increases the diversity of the deception detection network space, helping to discover better deception detection network structures than existing manually designed ones. This reduces the workload of manually designing deception detection network structures and improves the accuracy of deception detection.

[0052] 3. The present invention uses evidence depth technology to reduce the uncertainty of the lie detection network's lie detection results, and simultaneously gives the classification result and the uncertainty of the result for each input data, thereby improving the robustness of the lie detection network's prediction and the credibility of lie detection.

[0053] 4. The present invention adopts a multi-objective joint search strategy, taking the accuracy and uncertainty of the lie detection results as the goals of network optimization. The lie detection network model searched by this method performs excellently in both accuracy and uncertainty of the lie detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 Flow chart of the method of the present invention;

[0055] Figure 2 This is a diagram of the network architecture for lie detection provided by the present invention;

[0056] Figure 3 The optimal neural network structure selected for this invention Uncertainty measurement results diagram. DETAILED DESCRIPTION

[0057] In this embodiment, Figure 1 As shown in the figure, a video lie detection method based on capsule network structure search and uncertainty evaluation mainly uses the OpenFace tool and PyVHR tool to extract facial features and heart rate features from the video. Then, a network search space is composed of a channel attention layer, a pre-convolutional layer, a main capsule layer, and a digital capsule layer, and a network structure population is obtained by sampling. Finally, the accuracy and uncertainty are used as evaluation indicators to iteratively optimize the individual network structures in the network structure population, and finally the optimal lie detection network model is obtained for lie detection, as shown in the figure. Figure 1 As shown, the method is carried out in the following steps:

[0058] Step 1: Preprocess the video data to extract visual features and heart rate features;

[0059] Step 1.1 It is an open source facial recognition and facial expression analysis toolkit that provides a set of algorithms for face detection, facial feature extraction, and face recognition. Open source tools for any subject A video sample of Perform frame-level analysis and extraction to obtain the subject The facial feature matrix includes: Eye gaze direction matrix for the frame , No. Head pose matrix for the frame , No. Facial action unit matrix for a frame ;in, Indicates the Frame No. The eye gaze direction vector of each channel, Indicates the Frame No. Channel head pose vector, Indicates the Frame No. channels of facial action unit vectors, , , , , express The number of channels, express The number of channels, express The number of channels; F represents the video sample The total number of frames.

[0060] For video samples The corresponding category label is encoded to obtain the Category code value ;when When The category is lying, when hour, The category is not lying, that is , Indicates that the true category is lying, , Indicates the true category is not lying.

[0061] Step 1.2: The above video features mainly reflect external features, which are easy to disguise and thus interfere with lie detection. On the other hand, changes in physiological information (heart rate, etc.) are closely related to lying, and to a certain extent, they can objectively reflect whether the subject is lying. PyVHR (Python-based Video Heart Rate) is a Python-based library for contactless extraction of heart rate signals from videos. Use the CHROME heart rate extraction method in the PyVHR open source tool to extract heart rate signals from video samples. The heart rate signal vector is extracted and the data length of the heart rate signal vector is adjusted using the cubic spline interpolation method to obtain the heart rate feature vector. , Indicates the The heart rate feature value of the frame.

[0062] The self-collected data set used in this example is a lying experiment scenario designed by the present invention, which collects videos and heart rate data of 110 undergraduates and postgraduates, including 53 subjects who lied and 57 subjects who told the truth. Each subject recorded a video with an average video length of 371.9 seconds, the shortest video length was 217.5 seconds, and the longest video length was 471.8 seconds. In the self-collected data set, in this embodiment, the processing window length is set to 1 minute, and the sliding window method is used to process the entire video, and there is no overlap between adjacent windows. If the last window length is less than 1 minute, it will be discarded. For each 1-minute video clip Use the methods in steps 1.1 and 1.2 to extract the facial feature matrix and heart rate feature vector.

[0063] Step 1.3: According to the channel dimension Eye gaze direction matrix for the frame 、 Head pose matrix for the frame 、 Facial action unit matrix for a frame , heart rate feature vector Splice and get the lie detection feature matrix , After fusion, The lying detection feature vector of each channel, represents the number of channels of the lie detection feature matrix, and =M+N+O+1, .

[0064] Step 2: Construct a network space consisting of a channel attention layer, a pre-convolutional layer, a main capsule layer, and a digital capsule layer , the connection order of each layer is as follows Figure 2As shown; the channel attention layer includes: an efficient channel attention mechanism module and a selective kernel network module, and the serial number of any module in the channel attention layer is recorded as , where the number of branches of the selective kernel network module is 3, and the dimension of the feature descriptor z is 32; the pre-convolution layer includes: a two-layer convolution module, a three-layer convolution module, and a four-layer convolution module, and the serial number of any module in the pre-convolution layer is recorded as , where the two-layer convolution module is composed of a 1D convolutional neural network with a convolution kernel size of 5, a stride of 2, and a number of channels of 32 and 64 respectively; the three-layer convolution module is composed of a two-layer convolution module and a 1D convolutional neural network with a convolution kernel size of 5, a stride of 2, and a number of channels of 128; the four-layer convolution module is composed of a three-layer convolution module and a 1D convolutional neural network with a convolution kernel size of 1, a stride of 1, and a number of channels of 256; the main capsule layer includes: The main capsule modules with different output dimensions are recorded as , where the feature vector dimensions of the output capsule of the main capsule module are , =15; the digital capsule layer includes: digital capsule modules with different output dimensions, and the serial number of any module in the digital capsule layer is recorded as , where the feature vector dimensions of the digital capsule module output capsules are , =15; ; ; ; .

[0065] Step 3: Use uncertainty-based search algorithms to search from network space Search for the optimal network structure ;

[0066] Step 3.1, the current number of iterations is , the total number of iterations is , the total number of network individuals in the initial network structure pool is , the current network individual is ,initialization ;

[0067] A module is randomly selected from the channel attention layer, pre-convolution layer, main capsule layer and digital capsule layer to form the first Generation network structure population The network structure individuals, thus constructing the Generation network structure population in A network structure entity.

[0068] Step 3.2, Initialization ;

[0069] Step 3.3: Different feature channels often contain redundant and low-correlation information. In order to explore the contribution of heart rate features, video features, and different lie detection feature vectors to lie detection, the fused lie detection feature matrix In the input channel attention mechanism layer, the weights of each feature channel are redistributed. Enter Generation network structure population The In the individual network structures, the first Modules are processed to obtain Generation network structure population The Lie detection attention feature matrix ;

[0070] Will Enter Generation network structure population The The first one selected in the pre-convolutional layer of each network structure Modules are processed to obtain Generation network structure population The Lie detection pre-feature matrix ;

[0071] Will Enter Generation network structure population The The first one selected in the main capsule layer of the network structure individual Modules are processed to obtain Generation network structure population The lie detection main capsule feature matrix ;

[0072] Will Enter Generation network structure population The The first one selected in the digital capsule layer of the network structure individual The module is processed to optimize the relationship between the main capsule feature vector capsules in the main capsule feature matrix, improve the robustness and generalization of the model, and finally obtain the first Generation network structure population The Lie detection state capsule sequence , Indicates the Generation network structure population The The first one in the lie detection state capsule sequence Class state capsules, when hour, The state capsule representing the lying category, when hour, A state capsule representing the true category.

[0073] Step 3.4, calculation The L2 norm of the predicted category is obtained , and select the probability of lying prediction category represented by the larger L2 norm as the prediction category of video sample V; calculate the evidence of lying prediction category ,in, represents the rectified linear unit function.

[0074] Step 3.5, Build The The loss function of each individual network structure;

[0075] The boundary loss function is a specific loss function for capsule networks, which aims to encourage correct classification and appropriate spacing between state capsules. It promotes capsule networks to learn more discriminative feature representations by defining the boundary or spacing between targets and predictions and minimizing it. The uncertainty loss is a specific loss function for evidence-based deep learning, which aims to encourage the model to generate more evidence for each sample to reduce the uncertainty of the results and reduce the prediction variance when the generated evidence leads to better data fit.

[0076] Step 3.5.1: Use formula (1) to construct The The first Boundary loss function for each category :

[0077] (1)

[0078] In formula (1), yes If the category prediction is correct, then let =1, otherwise, let =0; represents the hyperparameter of the minimum distance between the correct and incorrect predicted classes and , is a hyperparameter representing the maximum distance between the correct and incorrect predicted classes and ; represents the weight that controls the degree of penalty for incorrectly predicted classes and .

[0079] Step 3.5.2: Use formula (1) to construct The The sum of the boundary loss functions of individual network structures ;

[0080] (2)

[0081] Step 3.5.3: Use formula (3) to construct The Uncertainty loss function of individual network structures :

[0082] (3)

[0083] In formula (3), It is Daidi The intensity parameter of the Dirichlet distribution of the individual network structures, and , is the annealing coefficient, yes Divergence, represents the Dirichlet distribution, express Middle The probability matrix of the lying prediction category assignment of the individual network structure is , represents the Dirichlet distribution parameter whose value is 1, express The The Dirichlet distribution parameter of the individual network structure is , .

[0084] Step 3.5.4: Use formula (4) to construct The The total loss function of each network structure ;

[0085] (5)

[0086] Step 3.6: Use the stochastic gradient descent algorithm to The Each network structure is trained individually, and the calculation To continuously adjust The weight parameters of the network structure network individuals until the maximum number of training times is reached So far, we can get the first A lie detection network entity.

[0087] Step 3.7, Input the training The predicted categories are used to calculate the first Daidi The accuracy rate is used as the The first evaluation index of a lie detection network individual ;

[0088] Evidence deep learning is a method that combines deep learning and evidence theory, aiming to improve the uncertainty and reliability estimation of the model. It has the characteristics of low computational complexity and no need for repeated network training. The uncertainty calculation formula of evidence deep learning in formula (5) is used to calculate the first Daidi uncertainty , and As the first The second evaluation index of the lie detection network individuals ;

[0089] (5)

[0090] Step 3.8, let Assign to After that, judge Is it established? If it is established, it means that the After generation training lie detection network individuals and constitute the first Generation of lie detection network populations and the two evaluation indicators of each lie detection network individual, and execute step 3.9; otherwise, return to step 3.3 and execute sequentially.

[0091] Step 3.9, according to the trained Two evaluation indicators for each individual in the lie detection network Each lie detection network individual performs non-dominated sorting and calculates the crowding distance, thereby selecting the one with the largest crowding distance on the Pareto front. A lie detection network individual, and as the first Generation population in A network structure individual;

[0092] Step 3.10, randomly select one module from the channel attention layer, pre-convolution layer, main capsule layer and digital capsule layer, and form the same Generation population A network structure individual with different individual structures, thus obtaining the first Generation network structure population In addition A network structure entity.

[0093] Step 3.11, let Assign to After that, judge Is it true? If so, it means that the T-th generation population is obtained. , and execute step 4, otherwise, return to step 3.1 and execute sequentially;

[0094] Step 3.12: Detect the lying network population based on the Tth generation Two evaluation indicators for each individual lie detection network in Perform non-dominated sorting on each lie detection network individual in the network, and select the lie detection network individual with the smallest index on the Pareto front as the optimal lie detection network .

[0095] Step 4: Enter the Optimal Lie Detection Network After processing and obtaining the optimal prediction category, the optimal total loss function is constructed according to formula (4) to be used for the optimal lie detection network Train to update The weight parameters are adjusted until the total number of training times is reached. So far, the optimal lie detection model after training is obtained to realize lie detection.

[0096] In this embodiment, an electronic device includes a memory and a processor. The memory is used to store a program that supports the processor to execute the above-mentioned video lie detection method. The processor is configured to execute the program stored in the memory.

[0097] In this embodiment, a computer-readable storage medium stores a computer program, which executes the steps of the above-mentioned video lying detection method when executed by a processor.

[0098] In practice, the dataset was fed into the trained deception model to obtain deception detection results and calculate their accuracy and uncertainty. The results were sorted by uncertainty from high to low, and results with high uncertainty were eliminated at a certain ratio, known as the rejection rate. The accuracy of the remaining results was calculated and plotted as a line graph. The optimal model, UCNAS, was compared with support vector machines (SVM), K-nearest neighbor (KNN), decision trees (DT), and capsule networks (DDCapsNet). The classification accuracy (ACC) and area under the curve (AUC) on the test set were used as evaluation metrics. The higher the recognition accuracy, the better the model's recognition ability. The classification results are shown in Table 1:

[0099] Table 1. Classification results of different methods on self-collected datasets

[0100]

[0101] The experimental results show that compared with SVM, KNN, DT, and DDCapsNet, the optimal lie detection model UCNAS selected by the proposed network structure search method has improved both the accuracy ACC and the area under the curve AUC classification indicators, which shows that UCNAS can classify samples more accurately in the lie detection task. Figure 3 As shown, this is the result of data screening based on the uncertainty of lying results in the present invention. The results show that this method can effectively measure the uncertainty of lying results.

[0102] In summary, the present invention integrates facial features and heart rate features extracted from videos, builds a network structure pool, inputs video data into network training, uses accuracy and uncertainty as network result evaluation indicators, uses the Pareto front surface to select the optimal model from the non-dominated solutions, and performs classification to achieve video-based automatic lie detection and uncertainty evaluation of lie results.

Claims

1. A video lie detection method based on capsule network structure search and uncertainty evaluation, characterized in that: The steps are as follows: Step 1: Extract video samples After concatenating the visual features and heart rate features, we get the lie detection feature matrix , After fusion, The lying detection feature vector of each channel, Indicates the number of channels of the lie detection feature matrix; Step 2: Construct a network space consisting of a channel attention layer, a pre-convolutional layer, a main capsule layer, and a digital capsule layer ; Wherein, the channel attention layer includes: an efficient channel attention mechanism module and a selective kernel network module, and the serial number of any module in the channel attention layer is recorded as The pre-convolution layer includes: a two-layer convolution module, a three-layer convolution module and a four-layer convolution module, and the serial number of any module in the pre-convolution layer is recorded as ; The main capsule layer comprises: The main capsule modules with different output dimensions are recorded as ; The digital capsule layer includes: digital capsule modules with different output dimensions, and the serial number of any module in the digital capsule layer is recorded as , ; ; ; ; Step 3: Use uncertainty-based search algorithms to search from network space Search for the optimal network structure ; Step 4: Enter the Optimal Lie Detection Network After processing and obtaining the optimal prediction category, the optimal total loss function is constructed according to formula (4) to be used for the optimal lie detection network Train to update The weight parameters are adjusted until the total number of training times is reached. So far, the optimal lie detection model after training is obtained to realize lie detection.

2. The video lying detection method based on capsule network structure search and uncertainty evaluation according to claim 1 is characterized in that: Step 1 is performed as follows: Step 1.1: For any subject A video sample of Perform frame-level analysis and extraction to obtain the subject The facial feature matrix includes: Eye gaze direction matrix for the frame , No. Head pose matrix for the frame , No. Facial action unit matrix for a frame ;in, Indicates the Frame No. The eye gaze direction vector of each channel, Indicates the Frame No. Channel head pose vector, Indicates the Frame No. channels of facial action unit vectors, , , , , express The number of channels, express The number of channels, express The number of channels; F represents the video sample The total number of frames; For the video sample The corresponding category label is encoded to obtain the Category code value ;when When The category is lying, when hour, The category is not lying; Step 1.2: From video sample Extract the heart rate signal vector and use the cubic spline interpolation method to adjust the data length of the heart rate signal vector to obtain the heart rate feature vector , Indicates the Heart rate feature value of the frame; Step 1.3: According to the channel dimension Eye gaze direction matrix for the frame 、 Head pose matrix for the frame 、 Facial action unit matrix for a frame , heart rate feature vector Splice and get the lie detection feature matrix , After fusion, The lying detection feature vector of each channel, represents the number of channels of the lie detection feature matrix, and =M+N+O+1, .

3. The video lying detection method based on capsule network structure search and uncertainty evaluation according to claim 2 is characterized in that: The step 3 is carried out as follows: Step 3.1, the current number of iterations is , the total number of iterations is , set the Generation network structure population The total number of individuals in the network structure is , the serial number of the current network individual is ,initialization ; A module is randomly selected from the channel attention layer, pre-convolution layer, main capsule layer and digital capsule layer to form the first Generation network structure population The network structure individuals, thus constructing the Generation network structure population in individual network structures; Step 3.2, Initialization ; Step 3.3, Enter Generation network structure population The In the individual network structures, the first Modules are processed to obtain Generation network structure population The Lie detection attention feature matrix ; Will Enter Generation network structure population The The first one selected in the pre-convolutional layer of each network structure Modules are processed to obtain Generation network structure population The Lie detection pre-feature matrix ; Will Enter Generation network structure population The The first one selected in the main capsule layer of the network structure individual Modules are processed to obtain Generation network structure population The lie detection main capsule feature matrix ; Will Enter Generation network structure population The The first one selected in the digital capsule layer of the network structure individual Modules are processed to obtain Generation network structure population The Lie detection state capsule sequence , Indicates the Generation network structure population The The first one in the lie detection state capsule sequence Class state capsules, when hour, The state capsule representing the lying category, when hour, State capsule representing the true category; Step 3.4, calculation The L2 norm of Predicted class probabilities of lying , and select the probability of the lying prediction category represented by the larger L2 norm as the prediction category of the video sample V; calculate the evidence of the lying prediction category ,in, represents the rectified linear unit function; Step 3.5, Build The The total loss function of each network structure ; Step 3.6: Use the stochastic gradient descent algorithm to The Each network structure is trained individually, and the calculation To continuously adjust The weight parameters of the network structure network individuals until the maximum number of training times is reached So far, we can get the first a lie detection network individual; Step 3.7, Input the training The predicted categories are used to calculate the first Daidi The accuracy rate is used as the The first evaluation index of a lie detection network individual , use formula (5) to calculate the Daidi uncertainty , and As the first The second evaluation index of the lie detection network individuals ; (5) Step 3.8, let Assign to After that, judge Is it established? If it is established, it means that the After generation training lie detection network individuals and constitute the first Generation of lie detection network populations and the two evaluation indicators of each lie detection network individual, and execute step 3.9; otherwise, return to step 3.3 and execute sequentially; Step 3.9, according to the trained Two evaluation indicators for individual lie detection networks Each lie detection network individual performs non-dominated sorting and calculates the crowding distance, thereby selecting the one with the largest crowding distance on the Pareto front. A lie detection network individual, and as the first Generation population in individual network structures; Step 3.10, randomly select one module from the channel attention layer, pre-convolution layer, main capsule layer and digital capsule layer, and form the same Generation population A network structure individual with different individual structures, thus obtaining the first Generation network structure population In addition individual network structures; Step 3.11, let Assign to After that, judge Is it true? If so, it means that the T-th generation population is obtained. , and execute step 4, otherwise, return to step 3.1 and execute sequentially; Step 3.12: Based on the T generation population Two evaluation indicators for each individual lie detection network in Perform non-dominated sorting on each lie detection network individual in the network, and select the lie detection network individual with the smallest index on the Pareto front as the optimal lie detection network .

4. The video lying detection method based on capsule network structure search and uncertainty evaluation according to claim 3 is characterized in that: Step 3.5 is performed as follows: Step 3.5.1: Use formula (1) to construct The The first Boundary loss function for each category : (1) In formula (1), yes If the category prediction is correct, then =1, otherwise, let =0; The hyperparameter representing the minimum distance between the correct and incorrect predicted categories, A hyperparameter representing the maximum distance between the correct and incorrect predicted classes; The weight that controls the degree of penalty for incorrectly predicted categories; Step 3.5.2: Use formula (2) to construct The The sum of the boundary loss functions of individual network structures ; (2) Step 3.5.3: Use formula (3) to construct The Uncertainty loss function of individual network structures : (3) In formula (3), It is Daidi The intensity parameter of the Dirichlet distribution of the individual network structures, and , is the annealing coefficient, yes Divergence, represents the Dirichlet distribution, express Middle The probability matrix of the lying prediction category assignment of the individual network structure is , represents the Dirichlet distribution parameter whose value is 1, express The The Dirichlet distribution parameter of the individual network structure is , ; express The The first Dirichlet distribution parameter for each category; Step 3.5.4: Use formula (4) to construct The The total loss function of each network structure ; (4)。 5. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the video lie detection method according to any one of claims 1 to 4, and the processor is configured to execute the program stored in the memory.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the video lie detection method according to any one of claims 1 to 4 are executed.

Images