A method for adjusting the detection frequency of a face authenticity detection algorithm

Abstract

This invention discloses a method for adjusting the detection frequency of a face authenticity detection algorithm, comprising the steps of: using a face authenticity detection algorithm to detect the authenticity of face images in video frames of a video stream; the detection frequency of the face authenticity detection algorithm adopts an initial frequency; extracting an audio segment from the video stream and obtaining a text sequence through a speech-to-text transcription model; using a semantic analysis model of a topic text classification network based on an ERNIE pre-trained model to analyze the text sequence and obtain a topic scene type, which is used as the current scene type of the video stream; obtaining a preset detection frequency based on the current scene type of the video stream; and updating the detection frequency of the face authenticity detection algorithm. This invention utilizes the results of scene analysis to adjust the detection frequency of the face authenticity detection algorithm, reducing the detection frequency in low-risk scenarios and increasing the detection frequency in high-risk scenarios, thereby improving the effective utilization of system resources.

Description

Technical Field

[0001] This invention relates to the field of Internet technology, and in particular to a method for adjusting the detection frequency of a face authenticity detection algorithm. Background Technology

[0002] With the widespread adoption of facial recognition technology, faces have become a crucial part of personal identity information. However, the privacy protection behind this technology is increasingly being challenged. For example, AI face-swapping technology has developed rapidly, but the improper use of these tools has led to numerous problems. Among these, face theft has seriously questioned the protection of personal privacy in face-swapping scenarios. On a smaller scale, deepfake technology poses many privacy issues; on a larger scale, it can easily influence politics and the economy, generating various impacts on public opinion. Because in many situations (such as remote interviews, live broadcasts), the face in a video is the only way to verify identity, face theft has become a potential security risk.

[0003] Deepfake face technology is currently mainly based on combinations and variations of relevant deep learning models, including Generative Adversarial Networks (GANs), Encoder-Decoder Networks, and Convolutional Neural Networks (CNNs). In particular, GANs play a crucial role in deepfakes, addressing the issue of low realism in generated fake faces. Utilizing these deep learning-based techniques, a well-designed deepfake face model can extract high-quality features from the original real face, representing them as vectors. Using these extracted feature vectors, it can generate fake faces with extremely high similarity to the original face, while maintaining a high degree of mobility, closely approximating the original real face and thus preventing privacy and defamation violations.

[0004] The rapid development of deepfake facial recognition technology has threatened the authenticity of faces, leading to its use by criminals in fraudulent scenarios such as video conferencing, causing serious consequences. However, some existing facial recognition algorithms struggle to balance real-time performance with system resource consumption. Some high-performing algorithms consume significant computing resources, making them unsuitable for local use by users or substantially increasing costs for service providers. Therefore, a balance and trade-off must be struck between various performance aspects to achieve better system resource utilization. Summary of the Invention

[0005] The purpose of this invention is to provide a method for adjusting the detection frequency of a face authenticity detection algorithm.

[0006] The technical solution for implementing the present invention is as follows:

[0007] A method for adjusting the detection frequency of a face authenticity detection algorithm, comprising the following steps:

[0008] A face authenticity detection algorithm is used to detect the authenticity of face images in video frames of the video stream; the detection frequency of the face authenticity detection algorithm adopts the initial frequency;

[0009] An audio segment is extracted from the video stream, and a text sequence is obtained through a speech-to-text transcription model;

[0010] The semantic analysis model of the topic text classification network based on the ERNIE pre-trained model is used to analyze the text sequence to obtain the topic scene type, which is used as the current scene type of the video stream.

[0011] Based on the current scene type of the video stream, and referring to a preset risk level table, the detection frequency for the corresponding risk level is obtained.

[0012] Used to update the detection frequency of the face authenticity detection algorithm.

[0013] In the preferred technical solution, the semantic analysis model using a topic text classification network based on an ERNIE pre-trained model analyzes the text sequence to obtain the topic scene type, which is then used as the current scene type of the video stream. Specifically:

[0014] Remove characters that cannot be represented in the text sequence and control characters other than newline, tab, and carriage return; replace newline, tab, and carriage return with spaces, and add spaces before and after Chinese characters; segment by spaces to obtain the smallest semantic unit token;

[0015] Each token is mapped to a Chinese character dictionary. The token is represented by its position in the dictionary, token_id. Tokens not included in the dictionary are replaced with [UNK].

[0016] Each token is converted into a one-dimensional 0-1 matrix of the dictionary size. That is, if the token_id of the token is 1, then the first digit of the resulting one-dimensional 0-1 matrix is ​​1, and the rest are 0; and so on.

[0017] Concatenate the one-dimensional 0-1 matrices obtained from converting all tokens to obtain a matrix of size equal to the number of tokens multiplied by the size of the dictionary. Then multiply this matrix by the embedding matrix to reduce its dimensionality and obtain vector X.

[0018] Vector X is input into a semantic analysis model of a topic text classification network based on an ERNIE pre-trained model. Vector X will pass through 6 encoder layers. In each encoder layer, vector X will first undergo a linear transformation to obtain 3 vectors: Q representing the input information features, K representing the content information features, and V representing the information itself.

[0019] , ;

[0020] In the formula, W represents the weight matrix, and T is the transpose symbol;

[0021] Q, K, and V are then processed through a multi-head attention mechanism with 12 heads to obtain the intermediate variable Z:

[0022] ,

[0023] ;

[0024] Where i is the header number, d k This is the scaling factor;

[0025] The intermediate variable Z then passes through the feed foreword network layer, and its result then enters the next layer:

[0026] ;

[0027] in, and Here are the model parameters, and W1 and W2 are the weight matrices in the feed-forward network layer;

[0028] The result Z after 6 layers of encoder transformation still needs to go through a linear transformation and softmax to finally obtain the output vector Y:

[0029] ;

[0030] The output vector Y contains the credibility of all topic scene types. The topic scene type with the highest credibility is selected as the current scene type.

[0031] This invention introduces a semantic analysis model for scene analysis, and uses the results of scene analysis to adjust the detection frequency of the face authenticity detection algorithm. In low-risk scenarios, the detection frequency is reduced to consume less computing resources, while in high-risk scenarios, the detection frequency is increased to utilize more computing resources for more effective detection, thereby improving the effective utilization rate of system resources. Attached Figure Description

[0032] Figure 1 This is a flowchart of an embodiment of the present invention. Detailed Implementation

[0033] This invention provides a method for adjusting the detection frequency of a face authenticity detection algorithm, comprising:

[0034] Step 1: Enter the specific scenario where face image authenticity detection is required and obtain the video stream, such as an online video conferencing scenario;

[0035] Step 2: Start the system and perform face authenticity detection based on a deep learning model at a certain initial frequency;

[0036] Step 3: Extract fixed-length audio information from the video stream at a fixed frequency for scene analysis and processing to obtain the specific topic scene of the current video stream, such as fourteen default scenes including entertainment, economy, and politics.

[0037] Step 4: Based on the specific topic scenarios obtained in Step 3, adjust the frequency of face image authenticity detection according to certain risk classification standards, thereby improving the system resource utilization rate.

[0038] In step 3, fixed-length audio information is extracted from the video stream at a fixed frequency for scene analysis processing, specifically as follows:

[0039] 3.1 Extract audio information of a certain appropriate length from the video stream;

[0040] 3.2 Input the extracted audio information into the speech-to-text model, and extract the speech content of the person in the video stream from the audio information as the text information to be processed;

[0041] 3.3 Input the obtained text information to be processed into the semantic analysis model. The model analyzes the text information to be processed to determine the current video content category and identify the topic scenario.

[0042] The specific method for obtaining specific topic scenarios through semantic analysis is as follows:

[0043] 3.3.1 Convert the input text to be processed into a vector using the word segmenter encoding;

[0044] 3.3.2 The obtained vector is fed into the semantic analysis model. The input vector will go through 6 encoder layers, and the final output vector will be obtained after going through 6 encoder layers.

[0045] 3.3.3 The output vector contains the credibility of 14 scene categories, thus obtaining the scene most likely to correspond to the current meeting.

[0046] In step 4, the specific risk levels of the topic scenarios will be classified as follows:

[0047] Level 1 Risk: Current Affairs;

[0048] Level 2 risks: finance, lottery, stocks;

[0049] Level 3 risks: education, technology, entertainment, games, sports, real estate, home furnishings, fashion, astrology, and society.

[0050] This invention introduces a semantic analysis model to complete the scene analysis function. This scene analysis function is used to adjust the frequency of face authenticity detection, thereby dynamically adjusting the detection frequency in scenarios requiring real-time detection. This enables dynamic allocation of system resources: reducing the detection frequency in low-risk scenarios to consume less computing resources, and increasing the detection frequency in high-risk scenarios to utilize more computing resources for more effective detection, thus improving the effective utilization of system resources. This invention also introduces a scene analysis function and provides an interface. By providing this interface, users can freely customize and use the scene analysis function, achieving more flexible and diverse system function customization and development. In this way, users can customize their own functional modules according to specific needs and scenarios, thus providing personalized solutions for different application scenarios.

[0051] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0052] A method for adjusting the detection frequency of a face authenticity detection algorithm involves first entering a video stream scenario, such as various real-time interactive video stream scenarios, and using a deep learning-based face authenticity detection model to detect the authenticity of faces in the video stream at a certain frequency. Then, a scene analysis function based on semantic analysis is combined to detect the risk level of the current video stream at a certain frequency. Based on the determined risk level, the default detection frequency is adjusted according to the risk level, thereby flexibly allocating computing resources and improving the system resource utilization rate.

[0053] Example

[0054] like Figure 1 As shown,

[0055] Step 1. Enter the video stream, i.e., online video conferencing, online video calls, online video presentations, etc. After entering the video stream, activate the face verification system described in this method to prepare for face verification of the people involved in the video stream.

[0056] Step 2. Extract video frames containing faces from the video stream at a certain frequency, and call the face authenticity detection algorithm based on the bidirectional dual-stream recurrent neural network deep learning model with special motion patterns of 68 specified key points of the face to identify, extract, and detect the authenticity of the face images in the video frames.

[0057] Specifically, this step includes:

[0058] Step 2.1 Use the open-source computer vision algorithm library OpenCV to process the video stream and extract video frames;

[0059] Step 2.2 uses the open-source machine learning algorithm library dlib library and calls the face_recognition function library to locate and crop the faces contained in the video frames, and obtains the cropped face frame sequence, which is convenient for subsequent authenticity detection processing;

[0060] Step 2.3 At a certain frequency that matches the frequency of the cropped face frame sequence, the face authenticity detection algorithm based on the bidirectional dual-stream recurrent neural network deep learning model with special motion patterns of 68 specified key points of the face is invoked to perform face image authenticity detection on the cropped face frame sequence of a certain time length obtained after processing in Step 2.2, and the face image authenticity detection result is obtained. The result is output according to the frequency to determine whether it is a real face.

[0061] Step 3. Extract a certain length of audio from the video stream at a certain frequency as an audio sequence to be processed for subsequent scene analysis and processing.

[0062] Specifically, this step includes:

[0063] Step 3.1 Confirm that the system has a usable audio recording device, or, as needed, enable a virtual audio recording device;

[0064] Step 3.2 Using the dshow framework in ffmpeg, use the device to record 30-second audio clips at a frequency of once per minute.

[0065] Step 3.3 Save the audio sequence information to be processed in WAV format with a sampling rate of 16K and a sampling depth of 16bit to ensure appropriate data sampling.

[0066] Step 4. Send the audio sequence information obtained in Step 3 to the Baidu API speech-to-text model to obtain a text sequence information containing audio content that can be processed by natural language processing, as the text information to be processed.

[0067] Step 5. Send the text information obtained in Step 4 to the semantic analysis model of the topic text classification network based on the ERNIE pre-trained model to further determine the topic scene type corresponding to the text information, which is the topic scene type corresponding to the audio sequence obtained in Step 3, and serve as the current scene type of the video stream.

[0068] Specifically, this step includes:

[0069] Step 5.1 First, remove characters that cannot be represented in the original input text information and control characters other than newline, tab, and carriage return. Then, replace newline, carriage return, tab and other whitespace characters with spaces. Add spaces before and after Chinese characters. Tokenize by spaces. Each two spaces are considered as a separate token, that is, the smallest semantic unit.

[0070] Step 5.2 Based on a dictionary containing 21,128 common Chinese characters, map each segmented token to the dictionary, and use the token_id in the dictionary to represent each token. If a token is not contained in the dictionary, use [UNK] instead.

[0071] Step 5.3 Convert each token_id into a one-dimensional 0-1 matrix of the dictionary size. If the id is 1, the first bit is 1 and the rest are 0.

[0072] Step 5.4 Multiply the set of 0-1 matrices corresponding to all tokens by an embedding matrix of size 2048 to reduce the dimensionality, and the final result is the input vector X;

[0073] Step 5.5 involves feeding the obtained vector into the model. The input vector X will pass through 6 encoder layers. In each encoder layer, vector X will first undergo a linear transformation to obtain 3 vectors: Q (query) representing the input information features, K (key) representing the content information features, and V (value) representing the information itself. The details of this step are as follows:

[0074]

[0075]

[0076]

[0077] In the formula, W represents the weight and T is the transpose sign;

[0078] Q, K, and V are then used to obtain the intermediate variable Z through a multi-head attention with 12 heads;

[0079] …,

[0080] The intermediate variable Z then passes through the feed foreword network layer, and its result then enters the next layer:

[0081]

[0082] in, and These are model parameters;

[0083] The result Z after 6 layers of encoder transformation still needs to go through a linear transformation and softmax to finally obtain the output vector Y:

[0084]

[0085] The output vector Y contains the credibility of each of the 14 scene categories (numbered 0-13 in sequence): politics, finance, lottery, stocks, education, technology, entertainment, games, sports, real estate, home, fashion, horoscope, and society. The scene type most likely to correspond to the current meeting is obtained based on the highest credibility among the 14 credibility values.

[0086] The form of Y is:

[0087]

[0088] This indicates the credibility of the scene category corresponding to number j.

[0089] Step 6. Using the current scene type of the video stream obtained in Step 5, compare it with the preset default risk level table to obtain the risk level and the corresponding face authenticity detection frequency.

[0090] Risk Level Table

[0091] Risk level Scene categories Detection frequency 1 Current Affairs 100ms / time 2 Finance, lottery, stocks 500ms / time 3 Education, technology, entertainment, games, sports, real estate, home furnishings, fashion, astrology, society 1000ms / time

[0092] Step 7. Using the authenticity detection frequency obtained in Step 6, adjust the face authenticity detection frequency to achieve flexible allocation of computing resources and improve system resource utilization.

Claims

1. A method for adjusting the detection frequency of a face authenticity detection algorithm, characterized in that, Including the following steps: A face authenticity detection algorithm is used to detect the authenticity of face images in video frames of the video stream; the detection frequency of the face authenticity detection algorithm adopts the initial frequency; An audio segment is extracted from the video stream, and a text sequence is obtained through a speech-to-text transcription model; The semantic analysis model of the topic text classification network based on the ERNIE pre-trained model is used to analyze the text sequence to obtain the topic scene type, which is used as the current scene type of the video stream. Based on the current scene type of the video stream, and referring to a preset risk level table, the detection frequency for the corresponding risk level is obtained. Used to update the detection frequency of the face authenticity detection algorithm.

2. The method for adjusting the detection frequency of the face authenticity detection algorithm as described in claim 1, characterized in that, The semantic analysis model using a topic text classification network based on an ERNIE pre-trained model analyzes the text sequence to obtain the topic scene type, which is then used as the current scene type of the video stream. Specifically: Remove characters that cannot be represented in the text sequence and control characters other than newline, tab, and carriage return; replace newline, tab, and carriage return with spaces, and add spaces before and after Chinese characters; By segmenting words using spaces, we obtain the smallest semantic unit, the token. Each token is mapped to a Chinese character dictionary. The token is represented by its position in the dictionary, token_id. Tokens not included in the dictionary are replaced with [UNK]. Each token is converted into a one-dimensional 0-1 matrix of the dictionary size. That is, if the token_id of the token is 1, then the first digit of the resulting one-dimensional 0-1 matrix is ​​1, and the rest are 0; and so on. Concatenate the one-dimensional 0-1 matrices obtained from converting all tokens to obtain a matrix of size equal to the number of tokens multiplied by the size of the dictionary. Then multiply this matrix by the embedding matrix to reduce its dimensionality and obtain vector X. Vector X is input into a semantic analysis model of a topic text classification network based on an ERNIE pre-trained model. Vector X will pass through 6 encoder layers. In each encoder layer, vector X will first undergo a linear transformation to obtain 3 vectors: Q representing the input information features, K representing the content information features, and V representing the information itself. ， ； In the formula, W represents the weight matrix, and T is the transpose symbol; Q, K, and V are then processed through a multi-head attention mechanism with 12 heads to obtain the intermediate variable Z: ， ； Where i is the header number, d k This is the scaling factor; The intermediate variable Z then passes through the feed foreword network layer, and its result then enters the next layer: ； in, and Here are the model parameters, and W1 and W2 are the weight matrices in the feed-forward network layer; The result Z after 6 layers of encoder transformation still needs to go through a linear transformation and softmax to finally obtain the output vector Y: ； The output vector Y contains the credibility of all topic scene types. The topic scene type with the highest credibility is selected as the current scene type.

Images