Video lie detection method based on capsule network structure search and uncertainty evaluation

Abstract

The invention discloses a video lie detection method based on capsule network structure search and uncertainty evaluation. The method comprises the following steps: 1, acquiring heart rate features and facial features from a video sample to form lie detection features; 2, constructing a channel attention layer, a front convolutional layer, a main capsule layer and a digital capsule layer, and sampling to generate a network structure population; 3, constructing a boundary loss function and an uncertainty loss function; 4, training each network structure individual in the network structure population, and generating a next generation of network structure population by taking the accuracy and the uncertainty as network structure individual evaluation indexes; 5, the network search process is repeated until the maximum number of iterations is reached, and therefore the optimal lie detection network individual is output; and 6, retraining the optimal lie detection network individual. According to the invention, lie-speaking detection can be carried out according to the lie-speaking detection features, the uncertainty of the detection result is evaluated, and a solution is provided for non-contact automatic lie-speaking detection based on videos.

Description

Technical Field

[0001] The present invention belongs to the technical field of biomedical signal processing, and in particular relates to a method for video lying detection and uncertainty evaluation of capsule network structure search. Background Art

[0002] Lying is becoming more and more common in people's daily interactions and may have serious consequences, including causing economic losses, undermining social trust, and moral corruption. Therefore, effective lie detection methods are crucial to maintaining social order. Currently, modalities such as video, audio, and text are widely used for lie detection. Among them, video contains rich multimodal information, such as facial expressions, body movements, and sounds, which makes it very useful for lie detection. At the same time, the acquisition of video data is becoming more and more convenient. People can easily record and share video content through various devices (such as smartphones, cameras, etc.), which provides a wide range of data sources and application scenarios for video lie detection. However, video lie detection is very challenging.

[0003] Although relevant deep learning technologies, especially convolutional neural networks, have shown outstanding advantages in lie detection. However, most existing network architectures are designed by human experts, which requires certain prior knowledge and experience, and is a time-consuming and error-prone process. In addition, due to people's inherent thinking patterns, it is difficult for experts to design an ideal model. Therefore, people hope to automatically search for network architectures through algorithms, which can greatly liberate the creativity of researchers and reduce the heavy cost of network design.

[0004] Since deception is rarely an outright lie, lie detection itself is uncertain. When an expert assesses whether a person is lying, he may come to a conclusion, but he is rarely 100% sure. Therefore, reasoning and decision-making about lying are uncertain, and uncertainty measurement of lie detection results is particularly important.

[0005] Therefore, how to automatically design a high-performance network based on deep learning technology and evaluate the uncertainty of the network's classification results is an urgent problem to be solved in video-based lie detection. Summary of the invention

[0006] The present invention addresses the deficiencies in the prior art and provides a video lie detection method based on capsule network structure search and uncertainty evaluation, aiming to reduce the uncertainty of lie detection results and improve the accuracy and credibility of lie detection.

[0007] The present invention adopts the following scheme to solve the technical problem:

[0008] The video lying detection method based on capsule network structure search and uncertainty evaluation of the present invention is characterized in that it is performed according to the following steps:

[0009] Step 1: Extract video samples After concatenating the visual features and heart rate features, we get the lie detection feature matrix , After fusion, The lying detection feature vector of channels, Indicates the number of channels of the lie detection feature matrix;

[0010] Step 2: Construct a network space consisting of a channel attention layer, a pre-convolutional layer, a main capsule layer, and a digital capsule layer ; Wherein, the channel attention layer includes: an efficient channel attention mechanism module and a selective kernel network module, and the serial number of any module in the channel attention layer is recorded as The pre-convolution layer includes: a two-layer convolution module, a three-layer convolution module and a four-layer convolution module, and the serial number of any module in the pre-convolution layer is recorded as ; The main capsule layer comprises: The main capsule modules with different output dimensions are recorded as ; The digital capsule layer comprises: digital capsule modules with different output dimensions, and the serial number of any module in the digital capsule layer is recorded as , ; ; ; ;

[0011] Step 3: Use uncertainty-based search algorithms to search for Search for the optimal network structure ;

[0012] Step 4: Enter the Optimal Lie Detection Network After the optimal prediction category is obtained, the optimal total loss function is constructed according to formula (4) to optimize the optimal lie detection network. Train to update The weight parameters are adjusted until the total number of training times is reached. So far, the optimal lie detection model after training is obtained, which is used to realize lie detection.

[0013] Further, step 1 is performed as follows:

[0014] Step 1.1. For any subject A video sample of Perform frame-level analysis and extraction to obtain the subject The facial feature matrix includes: Eye gaze direction matrix for the frame , No. Head pose matrix for the frame , No. Facial action unit matrix for a frame ;in, Indicates Frame No. The eye gaze direction vector of each channel, Indicates Frame No. The head pose vector of each channel, Indicates Frame No. channels of facial action unit vectors, , , , , express The number of channels, express The number of channels, express The number of channels; F represents the video sample The total number of frames;

[0015] For the video sample The corresponding category label is encoded to obtain Category code value ;when When The category is lying, when hour, The categories are not lying;

[0016] Step 1.2: From video samples The heart rate signal vector is extracted and the data length of the heart rate signal vector is adjusted using the cubic spline interpolation method to obtain the heart rate feature vector , Indicates The heart rate characteristic value of the frame;

[0017] Step 1.3: According to the channel dimension Eye gaze direction matrix for the frame , Head pose matrix for the frame , Facial action unit matrix for a frame , heart rate feature vector Splice and get the lie detection feature matrix , After fusion, The lying detection feature vector of channels, represents the number of channels of the lie detection feature matrix, and =M+N+O+1, .

[0018] Further, the step 3 is performed according to the following steps:

[0019] Step 3.1, the current number of iterations is , the total number of iterations is , set the Generation network structure population The total number of individuals in the network structure is , the serial number of the current network individual is ,initialization ;

[0020] A module is randomly selected from the channel attention layer, pre-convolution layer, main capsule layer and digital capsule layer to form the first Generation network structure population The network structure individuals, thus constructing the Generation network structure population In A network structure individual;

[0021] Step 3.2, Initialization ;

[0022] Step 3.3: Enter the Generation network structure population The In the network structure individuals, and after the first one selected in the channel attention layer modules to process and obtain Generation network structure population The A lying detection attention feature matrix ;

[0023] Will Enter the Generation network structure population The The first one selected in the pre-convolutional layer of each network structure Modules are processed to obtain Generation network structure population The Lie detection pre-feature matrix ;

[0024] Will Enter the Generation network structure population The The first selected in the main capsule layer of the network structure individual Modules are processed to obtain Generation network structure population The The main capsule feature matrix for lie detection ;

[0025] Will Enter the Generation network structure population The The selected Modules are processed to obtain Generation network structure population The Lie detection state capsule sequence , Indicates Generation network structure population The The first class state capsules, when hour, The state capsule representing the lying category, when hour, The state capsule representing the true category;

[0026] Step 3.4, calculation The L2 norm of Prediction probability of lying class , and select the probability of the lying prediction category represented by the larger L2 norm as the prediction category of the video sample V; calculate the evidence of the lying prediction category ,in, represents the rectified linear unit function;

[0027] Step 3.5: Build The The total loss function of each network structure ;

[0028] Step 3.6: Use the stochastic gradient descent algorithm to The Each network structure is trained individually and calculated. To continuously adjust The weight parameters of individual network structures until the maximum number of training times is reached So far, we can get the first A lie detection network individual;

[0029] Step 3.7: Input the training The predicted categories are used to calculate the first Daidi The accuracy rate is taken as The first evaluation index of a lie detection network individual , use formula (5) to calculate the Daidi Uncertainty , and As the The second evaluation index of the lie detection network individuals ;

[0030] (5)

[0031] Step 3.8, order Assign to After that, judge Is it true? If true, it means that the After generation training lie detection network individuals and constitute the first Generation of lie detection network populations and the two evaluation indicators of each individual of the lie detection network, and execute step 3.9; otherwise, return to step 3.3 and execute sequentially;

[0032] Step 3.9: According to the trained Two evaluation indicators for individual lie detection networks Each individual in the lying detection network performs non-dominated sorting and calculates the crowding distance, so as to select the one with the largest crowding distance on the Pareto front surface. A lie detection network individual, and as the first Generation population In A network structure individual;

[0033] Step 3.10: Randomly sample one module from the channel attention layer, pre-convolution layer, main capsule layer, and digital capsule layer, and form the same Generation population A network structure individual with different individual structures, thus obtaining the first Generation network structure population In addition A network structure individual;

[0034] Step 3.11, order Assign to After that, judge Is it true? If true, it means that the T-generation population is obtained. , and execute step 4, otherwise, return to step 3.1 and execute sequentially;

[0035] Step 3.12: According to the T generation population Two evaluation indicators for each individual lie detection network in Each lie detection network individual in the non-dominated sorting is performed, and the lie detection network individual with the smallest index on the Pareto front surface is selected as the optimal lie detection network .

[0036] Furthermore, step 3.5 is performed as follows:

[0037] Step 3.5.1: Use formula (1) to construct The The first Boundary loss function for each category :

[0038] (1)

[0039] In formula (1), yes If the category prediction is correct, then =1, otherwise, let =0; The hyperparameter representing the minimum distance between the correct predicted class and the wrong class prediction, A hyperparameter representing the maximum distance between the correct predicted class and the incorrect class prediction; The weight that controls the degree of penalty for incorrectly predicted categories;

[0040] Step 3.5.2: Use formula (2) to construct The The sum of the boundary loss functions of individual network structures ;

[0041] (2)

[0042] Step 3.5.3: Use formula (3) to construct The Uncertainty loss function of individual network structures :

[0043] (3)

[0044] In formula (3), It is Daidi The intensity parameter of the Dirichlet distribution of the individual network structures, and , is the annealing coefficient, yes Divergence, represents the Dirichlet distribution, express Middle The probability matrix of lying prediction category assignment of individuals in the network structure is , represents the Dirichlet distribution parameter whose value is 1, express The The Dirichlet distribution parameter of the individual network structure is , ; express The The first Dirichlet distribution parameters for the categories;

[0045] Step 3.5.4: Use formula (4) to construct The The total loss function of each network structure ;

[0046] (4).

[0047] The electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the video lie detection method, and the processor is configured to execute the program stored in the memory.

[0048] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program executes the steps of the video lying detection method when the computer program is executed by a processor.

[0049] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0050] 1. The present invention utilizes neural network architecture search technology to adaptively optimize the structure and parameters of the lie detection network, making the lie detection network model perform better, thereby significantly improving the accuracy of lie detection.

[0051] 2. The present invention also sets four search space modules, including a channel attention module, a pre-convolution module, a main capsule module, and a digital capsule module. The design of multiple modules increases the diversity of the lie detection network space, helps to find a lie detection network structure that is better than the existing manually designed lie detection network structure, thereby reducing the workload of manually designing the lie detection network structure and improving the accuracy of lie detection.

[0052] 3. The present invention adopts evidence depth technology to reduce the uncertainty of the lie detection results of the lie detection network, and gives the classification result and the uncertainty of the result for each input data, thereby improving the robustness of the lie detection network prediction and the credibility of lie detection.

[0053] 4. The present invention adopts a multi-objective joint search strategy, taking the accuracy and uncertainty of the lie detection results as the goals of network optimization. The lie detection network model searched by this method performs excellently in both accuracy and uncertainty of the lie detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flow chart of the method of the present invention;

[0055] Figure 2 A diagram of the network architecture of lying detection provided by the present invention;

[0056] Figure 3 The optimal neural network structure selected for the present invention Uncertainty measurement results diagram. DETAILED DESCRIPTION

[0057] In this embodiment, Figure 1 As shown in FIG, a video lie detection method based on capsule network structure search and uncertainty evaluation mainly uses the OpenFace tool and the PyVHR tool to extract facial features and heart rate features from the video, and then constructs a channel attention layer, a pre-convolutional layer, a main capsule layer, and a digital capsule layer to form a network search space and sample to obtain a network structure population. Finally, the accuracy and uncertainty are used as evaluation indicators to iteratively optimize the network structure individuals in the network structure population, and finally the optimal lie detection network model is obtained for lie detection, as shown in FIG. Figure 1 As shown, the method is carried out in the following steps:

[0058] Step 1: Preprocess video data to extract visual features and heart rate features;

[0059] Step 1.1 It is an open source facial recognition and facial expression analysis toolkit that provides a set of algorithms for face detection, facial feature extraction, and face recognition. Open source tools for any subject A video sample of Perform frame-level analysis and extraction to obtain the subject The facial feature matrix includes: Eye gaze direction matrix for the frame , No. Head pose matrix for the frame , No. Facial action unit matrix for a frame ;in, Indicates Frame No. The eye gaze direction vector of each channel, Indicates Frame No. The head pose vector of each channel, Indicates Frame No. channels of facial action unit vectors, , , , , express The number of channels, express The number of channels, express The number of channels; F represents the video sample The total number of frames.

[0060] For video samples The corresponding category label is encoded to obtain Category code value ;when When The category is lying, when hour, The category is not lying, that is , Indicates that the true category is lying, , Indicates the true category as not lying.

[0061] Step 1.2: The above video features mainly reflect external features, which are easy to disguise and thus interfere with lie detection. On the other hand, changes in physiological information (heart rate, etc.) are closely related to lying, which can objectively reflect whether the subject is lying to a certain extent. PyVHR (Python-based Video Heart Rate) is a Python-based library for non-contact extraction of heart rate signals from videos. Use the CHROME heart rate extraction method in the PyVHR open source tool to extract heart rate signals from video samples. The heart rate signal vector is extracted from the vector and the data length of the heart rate signal vector is adjusted using the cubic spline interpolation method to obtain the heart rate feature vector. , Indicates The heart rate feature value of the frame.

[0062] A self-collected data set is used in this example. The present invention collects video and heart rate data of 110 undergraduate and graduate students by designing a lying experiment scenario, including 53 subjects who lied and 57 subjects who told the truth. Each subject recorded a video with an average video length of 371.9 seconds, the shortest video length of 217.5 seconds, and the longest video length of 471.8 seconds. In the self-collected data set, in this embodiment, the processing window length is set to 1 minute, and the entire video is processed in a sliding window manner, and there is no overlap between adjacent windows. If the last window length is less than 1 minute, it will be discarded. For each 1-minute video clip Use the methods in steps 1.1 and 1.2 to extract the facial feature matrix and heart rate feature vector.

[0063] Step 1.3: According to the channel dimension Eye gaze direction matrix for the frame , Head pose matrix for the frame , Facial action unit matrix for a frame , heart rate feature vector Splice and get the lie detection feature matrix , After fusion, The lying detection feature vector of channels, represents the number of channels of the lie detection feature matrix, and =M+N+O+1, .

[0064] Step 2: Construct a network space consisting of a channel attention layer, a pre-convolutional layer, a main capsule layer, and a digital capsule layer The connection order of each layer is as follows Figure 2As shown; the channel attention layer includes: an efficient channel attention mechanism module and a selective kernel network module, and the serial number of any module in the channel attention layer is recorded as , where the number of branches of the selective kernel network module is 3, and the dimension of the feature descriptor z is 32; the pre-convolutional layer includes: a two-layer convolutional module, a three-layer convolutional module, and a four-layer convolutional module, and the serial number of any module in the pre-convolutional layer is recorded as , where the two-layer convolution module is composed of a 1D convolutional neural network with a convolution kernel size of 5, a stride of 2, and a number of channels of 32 and 64 respectively; the three-layer convolution module is composed of a two-layer convolution module and a 1D convolutional neural network with a convolution kernel size of 5, a stride of 2, and a number of channels of 128; the four-layer convolution module is composed of a three-layer convolution module and a 1D convolutional neural network with a convolution kernel size of 1, a stride of 1, and a number of channels of 256; the main capsule layer includes: The main capsule modules with different output dimensions are recorded as , where the feature vector dimensions of the output capsules of the main capsule module are , =15; the digital capsule layer includes: digital capsule modules with different output dimensions, and the serial number of any module in the digital capsule layer is recorded as , where the feature vector dimensions of the output capsules of the digital capsule module are , =15; ; ; ; .

[0065] Step 3: Use uncertainty-based search algorithms to search for Search for the optimal network structure ;

[0066] Step 3.1, the current number of iterations is The total number of iterations is , the total number of network individuals in the initial network structure pool is , the current network individual is ,initialization ;

[0067] A module is randomly selected from the channel attention layer, pre-convolution layer, main capsule layer and digital capsule layer to form the first Generation network structure population The network structure individuals, thus constructing the Generation network structure population In A network structure individual.

[0068] Step 3.2, Initialization ;

[0069] Step 3.3: Different feature channels often contain redundant and low-correlation information. In order to explore the contribution of heart rate features, video features and different lie detection feature vectors to lie detection, the fused lie detection feature matrix In the input channel attention mechanism layer, the weights of each feature channel are redistributed. Enter the Generation network structure population The In the network structure individuals, and after the first one selected in the channel attention layer Modules are processed to obtain Generation network structure population The A lying detection attention feature matrix ;

[0070] Will Enter the Generation network structure population The The first one selected in the pre-convolutional layer of each network structure Modules are processed to obtain Generation network structure population The Lie detection pre-feature matrix ;

[0071] Will Enter the Generation network structure population The The first selected in the main capsule layer of the network structure individual Modules are processed to obtain Generation network structure population The The main capsule feature matrix for lie detection ;

[0072] Will Enter the Generation network structure population The The selected The purpose of this module is to optimize the relationship between the main capsule feature vector capsules in the main capsule feature matrix, improve the robustness and generalization of the model, and finally obtain the first Generation network structure population The Lie detection state capsule sequence , Indicates Generation network structure population The The first class state capsules, when hour, The state capsule representing the lying category, when hour, State capsule representing the true category.

[0073] Step 3.4, calculation The L2 norm of the predicted category is obtained. , and select the probability of the lying prediction category represented by the larger L2 norm as the prediction category of the video sample V; calculate the evidence of the lying prediction category ,in, represents the rectified linear unit function.

[0074] Step 3.5: Build The The loss function of each individual network structure;

[0075] The boundary loss function is a specific loss function for capsule networks, which aims to encourage correct classification and appropriate spacing between state capsules. It drives the capsule network to learn more discriminative feature representations by defining the boundary or spacing between the target and the prediction and minimizing it. The uncertainty loss is a specific loss function for evidence deep learning, which aims to encourage the model to generate more evidence for each sample to reduce the uncertainty of the results and reduce the prediction variance when the generated evidence leads to better data fit.

[0076] Step 3.5.1: Use formula (1) to construct The The first Boundary loss function for each category :

[0077] (1)

[0078] In formula (1), yes If the category prediction is correct, then let =1, otherwise, let =0; represents the hyperparameter of the minimum distance between the correct and incorrect predictions and , is a hyperparameter representing the maximum distance between the correct and incorrect predictions and ; represents the weight that controls the degree of penalty for incorrectly predicted classes and .

[0079] Step 3.5.2: Use formula (1) to construct The The sum of the boundary loss functions of individual network structures ;

[0080] (2)

[0081] Step 3.5.3: Use formula (3) to construct The Uncertainty loss function of individual network structures :

[0082] (3)

[0083] In formula (3), It is Daidi The intensity parameter of the Dirichlet distribution of the individual network structures, and , is the annealing coefficient, yes Divergence, represents the Dirichlet distribution, express Middle The probability matrix of lying prediction category assignment of individuals in the network structure is , represents the Dirichlet distribution parameter whose value is 1, express The The Dirichlet distribution parameter of the individual network structure is , .

[0084] Step 3.5.4: Use formula (4) to construct The The total loss function of each network structure ;

[0085] (5)

[0086] Step 3.6: Use the stochastic gradient descent algorithm to The Each network structure is trained individually and calculated. To continuously adjust The weight parameters of individual network structures until the maximum number of training times is reached So far, we can get the first A lie detection network individual.

[0087] Step 3.7: Input the training The predicted categories are used to calculate the first Daidi The accuracy rate is taken as The first evaluation index of a lie detection network individual ;

[0088] Evidence deep learning is a method that combines deep learning and evidence theory, aiming to improve the uncertainty and reliability estimation of the model. It has the characteristics of small computational complexity and no need for repeated network training. The uncertainty calculation formula of evidence deep learning in formula (5) is used to calculate the first Daidi Uncertainty , and As the The second evaluation index of the lie detection network individuals ;

[0089] (5)

[0090] Step 3.8, order Assign to After that, judge Is it true? If true, it means that the After generation training lie detection network individuals and constitute the first Generation of lie detection network populations and the two evaluation indicators of each lie detection network individual, and execute step 3.9; otherwise, return to step 3.3 and execute sequentially.

[0091] Step 3.9: According to the trained Two evaluation indicators for individual lie detection networks Each individual in the lying detection network performs non-dominated sorting and calculates the crowding distance, so as to select the one with the largest crowding distance on the Pareto front surface. A lie detection network individual, and as the first Generation population In A network structure individual;

[0092] Step 3.10: Randomly sample one module from the channel attention layer, pre-convolution layer, main capsule layer, and digital capsule layer, and form the same Generation population A network structure individual with different individual structures, thus obtaining the first Generation network structure population In addition A network structure individual.

[0093] Step 3.11, order Assign to After that, judge Is it true? If true, it means that the T-generation population is obtained. , and execute step 4, otherwise, return to step 3.1 and execute sequentially;

[0094] Step 3.12: Detect the network population based on the Tth generation of lying Two evaluation indicators for each individual lie detection network in Each lie detection network individual in the non-dominated sorting is performed, and the lie detection network individual with the smallest index on the Pareto front surface is selected as the optimal lie detection network .

[0095] Step 4: Enter the Optimal Lie Detection Network After the optimal prediction category is obtained, the optimal total loss function is constructed according to formula (4) to optimize the optimal lie detection network. Train to update The weight parameters are adjusted until the total number of training times is reached. So far, the optimal lie detection model after training is obtained, which is used to realize lie detection.

[0096] In this embodiment, an electronic device includes a memory and a processor. The memory is used to store a program that supports the processor to execute the above-mentioned video lie detection method, and the processor is configured to execute the program stored in the memory.

[0097] In this embodiment, a computer-readable storage medium stores a computer program on the computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned video lying detection method are executed.

[0098] In the specific implementation, the data set is input into the trained lying model to obtain the lying detection results and calculate the accuracy and uncertainty. The results are sorted from high to low according to the uncertainty, and the results with high uncertainty are eliminated according to a certain ratio, which is called the rejection rate Reject Rate. The accuracy of the remaining results is calculated and plotted into a line graph. The selected optimal model UCNAS is compared with the support vector machine SVM, K nearest neighbor KNN, decision tree DT, and capsule network DDCapsNet. The classification accuracy ACC and area under the curve AUC on the test set are used as evaluation indicators. The higher the recognition accuracy, the better the recognition ability of the model. The classification results are shown in Table 1:

[0099] Table 1. Classification results of different methods on self-collected datasets

[0100]

[0101] The experimental results show that compared with SVM, KNN, DT, and DDCapsNet, the optimal lie detection model UCNAS selected by the proposed network structure search method has improved both accuracy ACC and area under the curve AUC classification indicators, which shows that UCNAS can classify samples more accurately in the lie detection task. Figure 3 As shown, it is the result of data screening based on the uncertainty of lying results in the present invention, and the result shows that the present method can effectively measure the uncertainty of lying results.

[0102] In summary, the present invention integrates facial features and heart rate features extracted from videos, builds a network structure pool, inputs video data into network training, uses accuracy and uncertainty as network result evaluation indicators, uses the Pareto front surface to select the optimal model from non-dominated solutions, and performs classification to achieve video-based automatic lie detection and uncertainty evaluation of lie results.

Claims

1. A video lying detection method based on capsule network structure search and uncertainty evaluation, characterized in that: The steps are as follows: Step 1: Extract video samples After concatenating the visual features and heart rate features, we get the lie detection feature matrix , After fusion, The lying detection feature vector of channels, Indicates the number of channels of the lie detection feature matrix; Step 2: Construct a network space consisting of a channel attention layer, a pre-convolutional layer, a main capsule layer, and a digital capsule layer ; Wherein, the channel attention layer includes: an efficient channel attention mechanism module and a selective kernel network module, and the serial number of any module in the channel attention layer is recorded as The pre-convolution layer includes: a two-layer convolution module, a three-layer convolution module and a four-layer convolution module, and the serial number of any module in the pre-convolution layer is recorded as ; The main capsule layer comprises: The main capsule modules with different output dimensions are recorded as ; The digital capsule layer comprises: digital capsule modules with different output dimensions, and the serial number of any module in the digital capsule layer is recorded as , ; ; ; ; Step 3: Use uncertainty-based search algorithms to search for Search for the optimal network structure ; Step 4: Enter the Optimal Lie Detection Network After the optimal prediction category is obtained, the optimal total loss function is constructed according to formula (4) to optimize the optimal lie detection network. Train to update The weight parameters are adjusted until the total number of training times is reached. So far, the optimal lie detection model after training is obtained, which is used to realize lie detection.

2. According to claim 1, a video lying detection method based on capsule network structure search and uncertainty evaluation is characterized in that: Step 1 is performed as follows: Step 1.

1. For any subject A video sample of Perform frame-level analysis and extraction to obtain the subject The facial feature matrix includes: Eye gaze direction matrix for the frame , No. Head pose matrix for the frame , No. Facial action unit matrix for a frame ;in, Indicates Frame No. The eye gaze direction vector of each channel, Indicates Frame No. The head pose vector of each channel, Indicates Frame No. channels of facial action unit vectors, , , , , express The number of channels, express The number of channels, express The number of channels; F represents the video sample The total number of frames; For the video sample The corresponding category label is encoded to obtain Category code value ;when When The category is lying, when hour, The categories are not lying; Step 1.2: From video samples The heart rate signal vector is extracted and the data length of the heart rate signal vector is adjusted using the cubic spline interpolation method to obtain the heart rate feature vector , Indicates The heart rate characteristic value of the frame; Step 1.3: According to the channel dimension Eye gaze direction matrix for the frame , Head pose matrix for the frame , Facial action unit matrix for a frame , heart rate feature vector Splice and get the lie detection feature matrix , After fusion, The lying detection feature vector of channels, represents the number of channels of the lie detection feature matrix, and =M+N+O+1, .

3. The video lying detection method based on capsule network structure search and uncertainty evaluation according to claim 2 is characterized in that: The step 3 is carried out according to the following steps: Step 3.1, the current number of iterations is , the total number of iterations is , set the Generation network structure population The total number of individuals in the network structure is , the serial number of the current network individual is ,initialization ; A module is randomly selected from the channel attention layer, pre-convolution layer, main capsule layer and digital capsule layer to form the first Generation network structure population The network structure individuals, thus constructing the Generation network structure population In A network structure individual; Step 3.2: Initialization ; Step 3.3: Enter the Generation network structure population The In the network structure individuals, and after the first one selected in the channel attention layer Modules are processed to obtain Generation network structure population The Lie detection attention feature matrix ; Will Enter the Generation network structure population The The first one selected in the pre-convolutional layer of each network structure Modules are processed to obtain Generation network structure population The Lie detection pre-feature matrix ; Will Enter the Generation network structure population The The first selected in the main capsule layer of the network structure individual Modules are processed to obtain Generation network structure population The The main capsule feature matrix for lie detection ; Will Enter the Generation network structure population The The selected Modules are processed to obtain Generation network structure population The Lie detection state capsule sequence , Indicates Generation network structure population The The first class state capsules, when hour, The state capsule representing the lying category, when hour, The state capsule representing the true category; Step 3.4, calculation The L2 norm of Prediction probability of lying class , and select the probability of the lying prediction category represented by the larger L2 norm as the prediction category of the video sample V; calculate the evidence of the lying prediction category ,in, represents the rectified linear unit function; Step 3.5: Build The The total loss function of each network structure ; Step 3.6: Use the stochastic gradient descent algorithm to The Each network structure is trained individually and calculated. To continuously adjust The weight parameters of individual network structures until the maximum number of training times is reached So far, we can get the first A lie detection network individual; Step 3.7: Input the training The predicted categories are used to calculate the first Daidi The accuracy rate is taken as The first evaluation index of a lie detection network individual , use formula (5) to calculate the Daidi Uncertainty , and As the The second evaluation index of the lie detection network individuals ; (5) Step 3.8, order Assign to After that, judge Is it true? If true, it means that the After generation training lie detection network individuals and constitute the first Generation of lie detection network populations and the two evaluation indicators of each individual of the lie detection network, and execute step 3.9; otherwise, return to step 3.3 and execute sequentially; Step 3.9: According to the trained Two evaluation indicators for individual lie detection networks Each individual in the lying detection network performs non-dominated sorting and calculates the crowding distance, so as to select the one with the largest crowding distance on the Pareto front surface. A lie detection network individual, and as the first Generation population In A network structure individual; Step 3.10: Randomly sample one module from each of the channel attention layer, pre-convolution layer, main capsule layer, and digital capsule layer, and form the same Generation population A network structure individual with different individual structures, thus obtaining the first Generation network structure population In addition A network structure individual; Step 3.11, order Assign to After that, judge Is it true? If true, it means that the T-generation population is obtained. , and execute step 4, otherwise, return to step 3.1 and execute sequentially; Step 3.12: According to the T generation population Two evaluation indicators for each individual lie detection network in Each lie detection network individual in the non-dominated sorting is performed, and the lie detection network individual with the smallest index on the Pareto front surface is selected as the optimal lie detection network .

4. The video lying detection method based on capsule network structure search and uncertainty evaluation according to claim 3 is characterized in that: Step 3.5 is performed as follows: Step 3.5.1: Use formula (1) to construct The The first Boundary loss function for each category : (1) In formula (1), yes If the category prediction is correct, then =1, otherwise, let =0; The hyperparameter representing the minimum distance between the correct predicted class and the wrong class prediction, A hyperparameter representing the maximum distance between the correct predicted class and the incorrect class prediction; The weight that controls the degree of penalty for incorrectly predicted categories; Step 3.5.2: Use formula (2) to construct The The sum of the marginal loss functions of individual network structures ; (2) Step 3.5.3: Use formula (3) to construct The Uncertainty loss function of individual network structures : (3) In formula (3), It is Daidi The intensity parameter of the Dirichlet distribution of the individual network structures, and , is the annealing coefficient, yes Divergence, represents the Dirichlet distribution, express Middle The probability matrix of lying prediction category assignment for individuals in the network structure is , represents the Dirichlet distribution parameter whose value is 1, express The The Dirichlet distribution parameter of the individual network structure is , ; express The The first Dirichlet distribution parameters for the categories; Step 3.5.4: Use formula (4) to construct The The total loss function of each network structure ; (4)。 5. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the video lie detection method according to any one of claims 1 to 4, and the processor is configured to execute the program stored in the memory.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the video lying detection method according to any one of claims 1 to 4 are executed.

Images