Face counterfeit video recognition method and device, storage medium and electronic equipment

Abstract

The invention provides a face counterfeit video recognition method and device, a storage medium and electronic equipment, and relates to the technical field of computers, and the method comprises the steps: obtaining a to-be-recognized face video; performing frame difference processing on each adjacent video frame in the face video to obtain a plurality of frame difference pictures; dividing each frame difference picture into image blocks in N area ranges; forming an image block sequence by the image blocks in the same area range in each frame difference picture, and obtaining N image block sequences; processing the N image block sequences by using a pre-constructed recognition model to obtain a recognition result of the face video; the recognition result represents whether the face video is a forged video or not. By applying the method provided by the embodiment of the invention, the recognition accuracy of the face counterfeit video can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, storage medium, and electronic device for identifying fake face videos. Background Technology

[0002] Facial recognition technology has been deeply integrated into core business scenarios such as financial identity verification and transaction authorization, and its security is directly related to user assets and system credibility. With the breakthrough development of deepfake technologies such as generative adversarial networks and diffusion models, high-fidelity forged facial videos pose a serious challenge to financial security.

[0003] Existing methods for identifying spoofed faces in videos have limitations in analyzing the relationship between facial changes across different video frames. They cannot identify facial changes between different video frames, resulting in low accuracy. Summary of the Invention

[0004] The technical problem this application aims to solve is to provide a method, apparatus, storage medium, and electronic device for identifying spoofed face videos, which can improve the accuracy of spoofed face video recognition. The specific solution is as follows:

[0005] A method for identifying fake face videos includes:

[0006] Obtain the video of the face to be identified;

[0007] Frame difference processing is performed on each adjacent video frame in the face video to obtain multiple frame difference images;

[0008] Each frame difference image is divided into N image blocks of varying regions; N is a positive integer.

[0009] Image blocks within the same region in each of the frame difference images are grouped into an image block sequence to obtain N image block sequences;

[0010] The N image block sequences are processed using a pre-built recognition model to obtain the recognition result of the face video; the recognition result indicates whether the face video is a fake video.

[0011] Optionally, in the above method, obtaining the face video to be identified includes:

[0012] When a user logs in or makes a transaction, the system retrieves the facial video entered by the user.

[0013] Optionally, in the above method, processing the N image patch sequences using a pre-built recognition model to obtain the recognition result of the face video includes:

[0014] Obtain the block feature vector for each of the image block sequences;

[0015] Position feature embedding is performed on each of the block feature vectors;

[0016] The feature vectors of each block after embedding the position features are fused to obtain fused features;

[0017] The fused features are input into a pre-built recognition model to obtain the recognition result of the face video.

[0018] Optionally, in the above method, fusing the block feature vectors after embedding the positional features to obtain fused features includes:

[0019] Each block feature vector after embedding the location features is input into a preset multi-level attention module to obtain the output feature vector corresponding to each block feature vector;

[0020] The output feature vectors corresponding to each block feature vector are concatenated to obtain the concatenated features;

[0021] Linear projection is performed on the spliced ​​features to obtain the fused features.

[0022] Optionally, after obtaining the recognition result of the face video, the above method further includes:

[0023] If the recognition result indicates that the face video is a fake video, a prompt message is output, which indicates that the verification of the face video has failed.

[0024] A device for identifying fake face videos includes:

[0025] The acquisition unit is used to acquire the video of the face to be identified.

[0026] A frame difference unit is used to perform frame difference processing on each adjacent video frame in the face video to obtain multiple frame difference images;

[0027] A partitioning unit is used to divide each frame difference image into N image blocks of varying regions; N is a positive integer.

[0028] An execution unit is used to combine image blocks in the same region of each frame difference image into an image block sequence to obtain N image block sequences;

[0029] The recognition unit is used to process the N image block sequences using a pre-built recognition model to obtain the recognition result of the face video; the recognition result indicates whether the face video is a fake video.

[0030] Optionally, the acquisition unit in the aforementioned apparatus includes:

[0031] The acquisition subunit is used to acquire the user's registered facial video when the user performs a login operation or a transaction operation.

[0032] Optionally, in the aforementioned apparatus, the identification unit includes:

[0033] An execution subunit is used to obtain the block feature vector for each of the image block sequences;

[0034] A feature embedding subunit is used to embed positional features into each of the block feature vectors;

[0035] The fusion subunit is used to fuse the block feature vectors after the position feature is embedded to obtain fused features;

[0036] The recognition subunit is used to input the fused features into a pre-built recognition model to obtain the recognition result of the face video.

[0037] A storage medium comprising stored instructions, wherein, when the instructions are executed, the device on which the storage medium is located executes the face spoofing video identification method described above.

[0038] An electronic device includes a memory and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors as described above for identifying spoofed face videos.

[0039] This application provides a method, apparatus, storage medium, and electronic device for identifying spoofed face videos. The method includes: acquiring a face video to be identified; performing frame difference processing on each adjacent video frame in the face video to obtain multiple frame difference images; dividing each frame difference image into N image blocks within a region; where N is a positive integer; grouping the image blocks within the same region in each of the frame difference images into an image block sequence to obtain N image block sequences; processing the N image block sequences using a pre-built recognition model to obtain a recognition result for the face video; the recognition result indicates whether the face video is a spoofed video. Applying the method provided in this application can improve the accuracy of identifying spoofed face videos. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0041] Figure 1 A flowchart illustrating a method for identifying fake face videos provided in this application;

[0042] Figure 2 A flowchart illustrating the process of recognizing spoofed face videos provided in this application;

[0043] Figure 3 A schematic diagram of image block division provided in this application;

[0044] Figure 4 An architecture diagram of an encoder provided for this application;

[0045] Figure 5 A schematic diagram of the structure of a face spoofing video recognition device provided in this application;

[0046] Figure 6 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0049] This application provides a method for identifying fake face videos. This method can be applied to electronic devices, such as computers, smartphones, tablets, smart wearable devices, etc. The method flowchart is shown below. Figure 1 As shown, it specifically includes:

[0050] S101: Obtain the video of the face to be identified.

[0051] In some embodiments provided in this application, optionally, obtaining the face video to be identified includes:

[0052] When a user logs in or makes a transaction, the system retrieves the facial video entered by the user.

[0053] In this embodiment, the user's facial video can be captured by the camera of an electronic device.

[0054] S102: Perform frame difference processing on each adjacent video frame in the face video to obtain multiple frame difference images.

[0055] In this embodiment, adjacent frames in the face video are subtracted by subtracting the image of the previous frame from the image of the next frame. Since each frame is an image of the same size, only the pixel values ​​at the same position need to be subtracted directly. For color images, the RGB three channels are subtracted separately to obtain the frame difference image.

[0056] S103: Divide each frame difference image into N image blocks of varying regions; N is a positive integer.

[0057] In this embodiment, the frame difference image can be evenly divided into N image patch blocks, each patch being the same size.

[0058] S104: Combine the image blocks in the same region of each frame difference image into an image block sequence to obtain N image block sequences.

[0059] In this embodiment, for each image, the positions are marked from 1 to N according to the row order. From a temporal perspective, in each frame, patch blocks at the same position form a sequence, resulting in a sequence of N image patches.

[0060] S105: The N image block sequences are processed using a pre-built recognition model to obtain the recognition result of the face video; the recognition result indicates whether the face video is a fake video.

[0061] In this embodiment, the recognition model includes an attention-based sequence model transformer encoder and transformer decoder, and a multi-classification output softmax layer.

[0062] The method provided in the embodiments of this application can improve the accuracy of identifying fake face videos.

[0063] In some embodiments provided in this application, optionally, obtaining the face video to be identified includes:

[0064] When a user logs in or makes a transaction, the system retrieves the facial video entered by the user.

[0065] In some embodiments provided in this application, optionally, the step of processing the N image patch sequences using a pre-built recognition model to obtain the recognition result of the face video includes:

[0066] Obtain the block feature vector for each of the image block sequences;

[0067] Position feature embedding is performed on each of the block feature vectors;

[0068] The feature vectors of each block after embedding the position features are fused to obtain fused features;

[0069] The fused features are input into a pre-built recognition model to obtain the recognition result of the face video.

[0070] In some embodiments provided in this application, optionally, the step of fusing the block feature vectors after embedding the position features to obtain fused features includes:

[0071] Each block feature vector after embedding the location features is input into a preset multi-level attention module to obtain the output feature vector corresponding to each block feature vector;

[0072] The output feature vectors corresponding to each block feature vector are concatenated to obtain the concatenated features;

[0073] Linear projection is performed on the spliced ​​features to obtain the fused features.

[0074] In some embodiments provided in this application, optionally, after obtaining the recognition result of the face video, the method further includes:

[0075] If the recognition result indicates that the face video is a fake video, a prompt message is output, which indicates that the verification of the face video has failed.

[0076] See Figure 2This application provides a flowchart of a process for recognizing spoofed facial videos. In the financial sector, facial video verification is required for security during login, transfers, and other operations, requiring customers to perform actions such as blinking and opening their mouths. This application uses the customer's facial video as input. From a temporal perspective, adjacent frames in the video are subtracted, with the image of the later frame subtracting the image of the previous frame. Since each frame is an image of the same size, only the pixel values ​​at the same position need to be subtracted directly. For color images, the RGB three channels are subtracted separately. For ease of explanation, it is assumed that the number of subtracted images is M. The obtained frame difference images are then patched. Figure 3 As shown, for M difference images, they are uniformly divided into N patches, each patch being the same size, assuming a height of ph and a width of pw. For each image, the positions are marked from 1 to N according to the row order. From a temporal perspective, in each frame, patch blocks at the same position form a sequence, as shown by the arrows in the figure, resulting in a total of N sequences. These N sequences are then modeled.

[0077] In this embodiment, both temporal and spatial dimensions are processed simultaneously to model N sequences. A single sequence represents the temporal dimension, and the N positions represent the spatial dimension; specifically, a multi-input Transformer architecture can be employed, such as... Figure 4 As shown. Each input is a sequence at the same position, and there are N inputs in total. For each input, perform the following operations in sequence:

[0078] For each input sequence, the patch is first flattened into a single line. If it is a GRB color image, the three channels of the GRB are flattened into a single line in turn, and then concatenated into a single line. The flattened dimension is ph*pw. If it is an RGB image, the length of the image feature vector is 3*ph*pw, where 3 is the number of channels.

[0079] After flattening the patch block, first perform Patch Embedding, then perform dimensional projection, and finally add Positional Embedding. Here, the Positional Embedding for the same input is the same because the positions are the same.

[0080] The projected features with added position encoding are directly input into the MLA module to obtain the output Oi, where i=1,2,3,...N.

[0081] Multiple Oi are concatenated and then subjected to a linear projection to obtain the fused feature.

[0082] Optionally, the fused features can be sequentially input into the transformer encoder and transformer decoder, then passed through a softmax function to output probability values. If the probability of being identified as false is greater than 0.5, the verification fails; otherwise, the verification passes.

[0083] and Figure 1 Corresponding to the method described herein, embodiments of this application also provide a device for identifying spoofed face videos, used for... Figure 1 The specific implementation of the method is shown in the following structural diagram. Figure 5 As shown, it includes:

[0084] Acquisition unit 501 is used to acquire the face video to be identified;

[0085] The frame difference unit 502 is used to perform frame difference processing on each adjacent video frame in the face video to obtain multiple frame difference images.

[0086] The partitioning unit 503 is used to divide each frame difference image into N image blocks of varying regions; N is a positive integer.

[0087] Execution unit 504 is used to combine image blocks in the same region of each frame difference image into an image block sequence to obtain N image block sequences;

[0088] The recognition unit 505 is used to process the N image block sequences using a pre-built recognition model to obtain the recognition result of the face video; the recognition result indicates whether the face video is a fake video.

[0089] In one embodiment provided in this application, based on the above-described solution, optionally, the acquisition unit 501 includes:

[0090] The acquisition subunit is used to acquire the user's registered facial video when the user performs a login operation or a transaction operation.

[0091] In one embodiment provided in this application, based on the above-described scheme, optionally, the identification unit 505 includes:

[0092] An execution subunit is used to obtain the block feature vector for each of the image block sequences;

[0093] A feature embedding subunit is used to embed positional features into each of the block feature vectors;

[0094] The fusion subunit is used to fuse the block feature vectors after the position feature is embedded to obtain fused features;

[0095] The recognition subunit is used to input the fused features into a pre-built recognition model to obtain the recognition result of the face video.

[0096] The specific principles and execution processes of each unit and module in the face spoofing video recognition device disclosed in the above-described embodiments of this application are the same as those of the face spoofing video recognition method disclosed in the above-described embodiments of this application. Please refer to the corresponding parts of the face spoofing video recognition method provided in the above-described embodiments of this application, and they will not be repeated here.

[0097] This application embodiment also provides a storage medium, which includes stored instructions, wherein, when the instructions are executed, the device where the storage medium is located controls the execution of the above-described method for identifying spoofed face videos.

[0098] This application also provides an electronic device, the structural schematic diagram of which is shown below. Figure 6 As shown, it specifically includes a memory 601 and one or more instructions 602, wherein one or more instructions 602 are stored in the memory 601 and are configured to be executed by one or more processors 603 to perform the above-mentioned method for recognizing fake face videos.

[0099] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0100] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application program, server, or storage medium executing the operation of this invention, based on the prompt message.

[0101] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0102] It is understood that the above notification and user authorization process is merely illustrative and does not constitute a limitation on the implementation of the present invention. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present invention.

[0103] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0104] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0105] Finally, it should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0106] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0107] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0108] The above provides a detailed description of a method for identifying fake face videos provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for identifying spoofed face videos, characterized in that, include: Obtain the video of the face to be identified; Frame difference processing is performed on each adjacent video frame in the face video to obtain multiple frame difference images; Each frame difference image is divided into N image blocks of varying regions; N is a positive integer. Image blocks within the same region in each of the frame difference images are grouped into an image block sequence to obtain N image block sequences; The N image block sequences are processed using a pre-built recognition model to obtain the recognition result of the face video; the recognition result indicates whether the face video is a fake video.

2. The method according to claim 1, characterized in that, The acquisition of the face video to be identified includes: When a user logs in or makes a transaction, the system retrieves the facial video entered by the user.

3. The method according to claim 1, characterized in that, The process of processing the N image patch sequences using a pre-built recognition model to obtain the recognition result of the face video includes: Obtain the block feature vector for each of the image block sequences; Position feature embedding is performed on each of the block feature vectors; The feature vectors of each block after location feature embedding are fused to obtain fused features; The fused features are input into a pre-built recognition model to obtain the recognition result of the face video.

4. The method according to claim 3, characterized in that, The process of fusing the block feature vectors after embedding the positional features to obtain fused features includes: Each block feature vector after embedding the location features is input into a preset multi-level attention module to obtain the output feature vector corresponding to each block feature vector; The output feature vectors corresponding to each block feature vector are concatenated to obtain the concatenated features; Linear projection is performed on the spliced ​​features to obtain the fused features.

5. The method according to claim 1, characterized in that, After obtaining the recognition result of the face video, the process further includes: If the recognition result indicates that the face video is a fake video, a prompt message is output, which indicates that the verification of the face video has failed.

6. A device for identifying spoofed facial videos, characterized in that, include: The acquisition unit is used to acquire the video of the face to be identified. A frame difference unit is used to perform frame difference processing on each adjacent video frame in the face video to obtain multiple frame difference images; A partitioning unit is used to divide each frame difference image into N image blocks of varying regions; N is a positive integer. An execution unit is used to combine image blocks in the same region of each frame difference image into an image block sequence to obtain N image block sequences; The recognition unit is used to process the N image block sequences using a pre-built recognition model to obtain the recognition result of the face video; the recognition result indicates whether the face video is a fake video.

7. The apparatus according to claim 6, characterized in that, The acquisition unit includes: The acquisition subunit is used to acquire the user's registered facial video when the user performs a login operation or a transaction operation.

8. The apparatus according to claim 6, characterized in that, The identification unit includes: An execution subunit is used to obtain the block feature vector for each of the image block sequences; A feature embedding subunit is used to embed positional features into each of the block feature vectors; The fusion subunit is used to fuse the block feature vectors after the position feature is embedded to obtain fused features; The recognition subunit is used to input the fused features into a pre-built recognition model to obtain the recognition result of the face video.

9. A storage medium, characterized in that, The storage medium includes storage instructions, wherein, when the instructions are executed, the device where the storage medium is located is controlled to perform the face spoofing video recognition method as described in any one of claims 1 to 5.

10. An electronic device, characterized in that, It includes a memory and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors as described in any one of claims 1 to 5 for the identification of fake face videos.

Images