Video detection method and device, electronic equipment and storage medium

Abstract

The invention relates to a video detection method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a to-be-detected video in a process of displaying an original video by terminal equipment; the to-be-detected video is a video intercepted from an original video; acquiring face image data based on the to-be-detected video; the face image data comprises a face image extracted from a video frame included in the to-be-detected video; calling a face detection model, and detecting the face image data to obtain a detection result of the to-be-detected video; the face detection model is used for detecting whether the face in the face image is a fake face, and the detection result is used for representing whether the fake face exists in the to-be-detected video. According to the method, the face detection model is adopted to detect the face image data in real time, the detection effect and the detection accuracy are improved, the content irrelevant to the face in the to-be-detected video is removed from the face image data, interference information during detection is reduced under the condition that detection is not affected, and the detection efficiency is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a video detection method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, AI technology has the ability to generate highly realistic audio and video. While AI technology brings convenience to people's lives, it also brings a series of security risks, such as misleading information, false news, and fraud. For videos, it is important for users to determine whether the video is generated based on generative AI technology, or whether the video is fake.

[0003] In the related art, users can only use their eyes to identify whether a video is forged, but it is difficult for the human eye to detect some details in the video, and the recognition accuracy is not high, resulting in poor recognition effect and low recognition accuracy. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a video detection method, device, electronic device and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, a video detection method is provided, the method comprising: In the process of displaying the original video on the terminal device, obtaining a video to be detected; the video to be detected is a video intercepted from the original video; Based on the video to be detected, acquiring face image data; the face image data includes a face image extracted from a video frame included in the video to be detected; A face detection model is called to detect the face image data to obtain a detection result of the video to be detected; the face detection model is used to detect whether the face in the face image is a forged face, and the detection result is used to characterize whether there is a forged face in the video to be detected.

[0006] In some embodiments, the acquiring of facial image data based on the video to be detected includes: For each video frame in the video to be detected, extracting facial key points in the video frame, and extracting a facial image in the video frame based on the facial key points; The facial image data is determined based on the extracted at least one facial image.

[0007] In some embodiments, obtaining the video to be detected includes: The original video is intercepted using a preset sliding window to obtain the video to be detected; the preset sliding window is used to intercept a video of a preset length, and the video to be detected is a video clip of the preset length that is most recently displayed in the original video.

[0008] In some embodiments, the using a preset sliding window to intercept the original video to obtain the video to be detected includes: In response to a recording instruction for the original video, start recording the original video, and when the recording time reaches the preset time, stop recording to obtain the video to be detected; or, A video segment of the preset length is copied from the original video, and the copied video segment is used as the video to be detected; wherein the original video is a video decoded by a preset decoder or a video in a buffer queue.

[0009] In some embodiments, copying the video segment of the preset length from the original video includes: Based on a preset frame rate, a video segment of the preset length is copied from the original video; the preset frame rate is used to represent the number of video frames copied from the original video per second, and the preset frame rate is less than or equal to the video frame rate of the original video.

[0010] In some embodiments, the face detection model includes a feature extraction network and a classification network, and calling the face detection model to detect the face image data to obtain a detection result includes: Calling the feature extraction network to perform feature extraction on the face image data to obtain target features corresponding to the face image data; the target features are used to characterize features of the face in the face image data; The classification network is called to classify the target features to obtain the detection result.

[0011] In some embodiments, the training process of the face detection model includes: Acquire sample data, the sample data including a first sample video, a first label corresponding to the first sample video, a second sample video, and a second label corresponding to the second sample video, the first sample video being a video that does not contain a forged face, the first label being used to indicate that there is no forged face in the first sample video, the second sample video being a video that contains a forged face, and the second label being used to indicate that there is a forged face in the second sample video; Based on the sample data, an initial face detection model to be trained is trained to obtain the face detection model.

[0012] In some embodiments, the training of the initial face detection model to be trained based on the sample data to obtain the face detection model includes: Based on the first sample video, first sample face image data is obtained; the first sample face image data includes a face image extracted from a video frame included in the first sample video; Based on the second sample video, obtaining second sample face image data; the second sample face image data includes a face image extracted from a video frame included in the second sample video; Inputting the first sample face image data and the second sample face image data into the initial face detection model respectively, and obtaining a first predicted detection result of the first sample face image data and a second predicted detection result of the second sample face image data; Based on the difference between the first predicted detection result and the first label, and the difference between the second predicted detection result and the second label, the model parameters of the initial face detection model are adjusted to obtain the face detection model.

[0013] In some embodiments, the method further comprises: In response to the detection result indicating that a forged face exists in the video to be detected, a prompt message is issued, where the prompt message is used to prompt a target object that there is a risk of a forged face in the original video.

[0014] According to a second aspect of an embodiment of the present disclosure, a video detection device is provided, the device comprising: A first acquisition module is configured to acquire a video to be detected during the process of displaying an original video on a terminal device; the video to be detected is a video intercepted from the original video; A second acquisition module is configured to acquire face image data based on the video to be detected; the face image data includes a face image extracted from a video frame included in the video to be detected; The processing module is configured to call a face detection model to detect the face image data to obtain a detection result of the video to be detected; the face detection model is used to detect whether the face in the face image is a forged face, and the detection result is used to characterize whether there is a forged face in the video to be detected.

[0015] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including: processor; a memory for storing processor-executable instructions; The processor is configured to execute the video detection method as described in the first aspect of the embodiment of the present disclosure.

[0016] According to a fourth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the video detection method as described in the first aspect of the embodiment of the present disclosure.

[0017] The above method of the present disclosure has the following beneficial effects: The method provided by the embodiment of the present disclosure intercepts the video to be detected from the original video in real time during the process of the terminal device displaying the original video, and extracts the face image data including the face image based on the model to be detected, and then uses the face detection model to detect whether there is a fake face in the face image included in the face image data. Compared with the human eye recognition method in the related technology, the face detection model is used for real-time detection, which makes it easier to find some details in the face image and has high detection accuracy, thereby improving the detection effect and detection accuracy. In addition, by detecting the face image data, other content that is not related to the face in the video to be detected can be removed, and the interference information during detection can be reduced without affecting the detection, thereby improving the detection efficiency, saving time, and saving power consumption.

[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0020] Figure 1 is a flow chart of a video detection method according to an exemplary embodiment.

[0021] Figure 2 It is a schematic diagram showing a prompt message according to an exemplary embodiment.

[0022] Figure 3 It is a structural diagram of a face detection model according to an exemplary embodiment.

[0023] Figure 4 It is a schematic diagram of a model training process according to an exemplary embodiment.

[0024] Figure 5 is a schematic diagram of a video detection method according to an exemplary embodiment.

[0025] Figure 6 It is a block diagram of a video detection device according to an exemplary embodiment.

[0026] Figure 7 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0027] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0028] The video detection method provided by the embodiment of the present disclosure obtains the video to be detected during the process of the terminal device displaying the original video; the video to be detected is a video intercepted from the original video; based on the video to be detected, facial image data is obtained; the facial image data includes facial images extracted from the video frames included in the video to be detected; a face detection model is called to detect the facial image data to obtain the detection result of the video to be detected; the face detection model is used to detect whether the face in the video is a fake face, and the detection result is used to characterize whether there is a fake face in the video to be detected. Compared with the human eye recognition method in the related art, this method uses a face detection model for real-time detection, which makes it easier to find some details in the face image, and the detection accuracy is high, thereby improving the detection effect and detection accuracy, and, by detecting the facial image data, other contents unrelated to the face in the video to be detected can be removed, and the interference information during detection can be reduced without affecting the detection, thereby improving the detection efficiency, saving time, and saving power consumption.

[0029] The method provided by the embodiments of the present disclosure can be applied to a variety of scenarios, for example, to scenarios involving video display, such as video call scenarios and video viewing scenarios.

[0030] Taking the video call scenario as an example, when the user is on a video call, the terminal device displays the other party's call video (i.e., the original video) in real time, extracts the video to be detected from the call video, and obtains facial image data based on the video to be detected. The face detection model is then called to detect the facial image data to determine whether there is a fake face in the video to be detected. If there is a fake face in the video to be detected, it means that the other party's video image during the current video call may be synthesized by AI, and there is a high possibility that the current video call is subject to fraud risk. If there is no fake face in the video to be detected, it means that the other party's video image during the current video call is a normal face image, and there is no fraud risk or the possibility of fraud risk in the current video call is very low.

[0031] Taking the video watching scenario as an example, when the user is watching the video, the terminal device displays the video the user is watching (i.e., the original video) in real time, and extracts the video to be detected from the original video, and obtains facial image data based on the video to be detected, and then calls the face detection model to detect the facial image data to determine whether there is a fake face in the video to be detected. If there is a fake face in the video to be detected, it means that the video currently being watched may be synthesized by AI, and the video currently being watched may be a fake video. If there is no fake face in the video to be detected, it means that the video currently being watched is a normal face picture, and the possibility of the video currently being watched being a real video or a fake video is very low.

[0032] Of course, it can also be applied to other video display scenarios, and the embodiments of the present disclosure do not limit this.

[0033] The video detection method provided in the embodiment of the present disclosure is executed by an electronic device, which may be a terminal device, such as a mobile phone, a tablet computer, a notebook, an intelligent robot, an intelligent wearable device, etc. The electronic device may also be a cloud device, such as a cloud server, etc.

[0034] Figure 1 is a flowchart of a video detection method according to an exemplary embodiment, which is executed by an electronic device, see Figure 1 The method includes the following steps S101-S103.

[0035] Step S101, while the terminal device is displaying the original video, a video to be detected is obtained; the video to be detected is a video captured from the original video.

[0036] The original video is the video currently displayed in real time by the terminal device. The original video can be the call video of the other party when the user is currently using the terminal device to make a video call with other users, or it can be the video currently being played by the terminal device; the video to be detected is a video intercepted from the original video, and the video to be detected includes all or part of the video frames in the original video, that is, all or part of the video frames in the original video can be used as the video to be detected. For example, the video to be detected can include the video frames within the preset time length that are most recently displayed in the original video, or the preset number of video frames that are most recently displayed in the original video.

[0037] Step S102, based on the video to be detected, acquiring face image data; the face image data includes a face image extracted from a video frame included in the video to be detected.

[0038] Among them, the facial image data includes at least one facial image. When the facial image data includes multiple facial images, the facial image data can be a video or a set consisting of multiple facial image data. Regardless of the form of the facial image data, the multiple facial images in the facial image data need to be arranged in the chronological order in the video to be detected.

[0039] In some embodiments, for each video frame in the video to be detected, facial key points in the video frame are extracted, and based on the facial key points, facial images in the video frame are extracted; and facial image data is determined based on at least one extracted facial image.

[0040] Optionally, a key point detection model may be used to determine and extract key points of the face in the video frame, or other methods may be used to determine key points of the face in the video frame. After the key points of the face are determined, the face region in the video frame may be determined based on the position of the key points of the face in the face, thereby extracting the face region (i.e., face image). Of course, other methods may also be used to extract face images from video frames, and the embodiments of the present disclosure are not limited to this.

[0041] It should be noted that if no facial image is extracted based on the video to be detected, that is, no facial image data is obtained, the subsequent detection steps will no longer be performed.

[0042] Step S103, calling the face detection model to detect the face image data to obtain the detection result of the video to be detected; the face detection model is used to detect whether the face in the face image is a fake face, and the detection result is used to indicate whether there is a fake face in the video to be detected.

[0043] The face detection model is a pre-trained model used to detect whether the face in the face image is a fake face. A fake face refers to a false face generated by generative AI technology, or a face obtained by synthesizing one person's face onto another person's face. When the face image data includes multiple face images, if it is detected that the face in at least one face image in the face image data is a fake face, then the detection result indicates that there is a fake face in the video to be detected. If it is detected that the faces in all face images in the face image data are not fake faces, then the detection result indicates that there is no fake face in the video to be detected.

[0044] It should be noted that the above steps S101-S102 can be performed by the terminal device or by the cloud device. When the face detection model is deployed on the terminal device, the above steps S101-S102 are performed by the cloud device. When the face detection model is deployed on the cloud device, the above steps S101-S102 are performed by the cloud device.

[0045] In some embodiments, if executed by a cloud device, the cloud device obtaining the video to be detected includes: the terminal device sends the video to be detected to the cloud device, and the cloud device receives the video to be detected sent by the terminal device; or, the terminal device sends the original video to the cloud device, the cloud device receives the original video sent by the terminal device, and then cuts out the video to be detected from the original video. In order to enable the user to understand the detection result, the cloud device can also send the detection result to the terminal device after obtaining the detection result.

[0046] Compared with the human eye recognition method in the related art, the method provided by the embodiment of the present disclosure adopts a face detection model for real-time detection, which is easier to find some details in the face image and has high detection accuracy, thereby improving the detection effect and detection accuracy. In addition, by detecting the face image data, other content unrelated to the face in the video to be detected can be removed, and interference information during detection can be reduced without affecting the detection, thereby improving the detection efficiency, saving time, and saving power consumption.

[0047] In some embodiments, in order to ensure the real-time performance and detection speed during detection, the duration of the video to be detected is not suitable to be too long. In this case, the terminal device obtains the video to be detected, including: the terminal device uses a preset sliding window to intercept the original video to obtain the video to be detected, wherein the preset sliding window is used to intercept the video of the preset duration, and the duration of the video to be detected is the video clip of the preset duration that is most recently displayed in the original video. In this way, in the process of the terminal device displaying the original video, the terminal device can intercept at least one video to be detected from the original video, and whenever a video to be detected is intercepted, it is detected based on the video to be detected, thereby ensuring that the detection process has higher real-time performance and faster detection speed. For example, the preset duration is 5 seconds, then the latest 5-second video clip is intercepted from the original video each time as the video to be detected. Optionally, if the duration of the original video is less than the preset duration, the original video is directly used as the video to be detected.

[0048] Optionally, the terminal device uses a preset sliding window to capture the original video to obtain the video to be detected, including at least one of the following first method and second method.

[0049] The first method: in response to a recording instruction for an original video, start recording the original video, and when the recording time reaches a preset time, stop recording to obtain a video to be detected.

[0050] The above-mentioned process of recording the video to be detected is any recording process. When the terminal device displays the original video, the terminal device triggers a recording instruction every preset time period from the beginning of displaying the original video until the original video display ends, thereby obtaining at least one video to be detected.

[0051] The second method is to copy a video segment of a preset length from the original video and use the copied video segment as the video to be detected; wherein the original video is a video decoded by a preset decoder or a video in a buffer queue.

[0052] Optionally, a preset function is inserted into a preset decoder, and based on the preset function, a video segment of a preset length is copied from the original video decoded by the preset decoder, and the copied video segment is used as the video to be detected. The preset function may be a hook function.

[0053] Optionally, a preset function is inserted into the buffer queue, and based on the preset function, a video segment of a preset length is copied from the original video waiting to be displayed in the buffer queue, and the copied video segment is used as the video to be detected.

[0054] For the terminal device, when displaying the original video, if the original video is not a video decoded by a preset decoder, the method of inserting a preset function in the preset decoder to copy the video segment cannot be used, and no matter what image or video is displayed on the screen of the terminal device, it needs to pass through the buffer queue in the terminal device. Therefore, the method of inserting a preset function in the buffer queue to copy the copy segment has a wider range of application. In the video detection process, a suitable method can be selected according to actual needs, and the embodiment of the present disclosure does not limit this.

[0055] Optionally, copying a video segment of a preset length from the original video includes: copying a video segment of a preset length from the original video based on a preset frame rate. The preset frame rate is used to characterize the number of video frames copied from the original video per second, and the preset frame rate is less than or equal to the video frame rate of the original video. For example, the preset frame rate may be 1fps (Frames Per Second, the number of frames transmitted per second), 2fps, the video frame rate of the original video, or other frame rates. In the actual process of video detection, a suitable preset frame rate may be set according to actual needs, such as the throughput, performance, power consumption load of the face detection model, and the real-time requirements for video detection, and the embodiments of the present disclosure do not impose any restrictions on this.

[0056] In addition, it should be noted that the above-mentioned methods of obtaining the video to be detected are methods in which the terminal device intercepts the video to be detected from the original video. If the cloud device intercepts the video to be detected from the original video, the cloud device can directly intercept the video to be detected from the received original video. Compared with the method in which the terminal device intercepts the video to be detected in real time, if the cloud device intercepts the video to be detected, the real-time performance of the video to be detected will be reduced. However, for the terminal device, the terminal device can reduce some operations, which is beneficial to reducing the power consumption of the terminal device. If the cloud device executes the subsequent steps of calling the face detection model for detection, it can be selected according to actual needs whether the terminal device intercepts the video to be detected from the original video or the cloud device intercepts the video to be detected from the original video. The disclosed embodiments do not impose any restrictions on this.

[0057] In some embodiments, after obtaining the detection result, the terminal device needs to feed back the detection result to the user. In response to the detection result indicating that there is a fake face in the video to be detected, the terminal device issues a prompt message, which is used to prompt the target object that there is a risk of a fake face in the original video. The target object is the user currently using the terminal device.

[0058] Optionally, the prompt message may be a text message, such as a text message displayed on a display screen saying “This call may involve a fake face, please be cautious in deciding whether to hang up”, or a voice message, such as a voice message saying “This call may involve a fake face, please be cautious in deciding whether to hang up”. Figure 2 The schematic diagram of a prompt information shown in FIG. 1 is a schematic diagram of a prompt information, where the prompt information can be displayed on the display screen of the terminal device in the form of a pop-up window, and at the same time, an operation button can be displayed in the pop-up window, such as Figure 2 The operation buttons of "Ignore" and "Hang up" are shown for the user to select.

[0059] For the user, the prompt information sent by the terminal device is only used to remind the user that there may be fake faces in the original video currently displayed. When the user receives the prompt information, the user can perform corresponding operations based on the actual situation.

[0060] In some embodiments, for a face detection model, the face detection model includes a feature extraction network and a classification network, calling the face detection model, detecting face image data, and obtaining a detection result, including: calling the feature extraction network, extracting features from the face image data, and obtaining target features corresponding to the video to be detected, the target features are used to characterize the features of the face in the video to be detected; calling the classification network, classifying the target features, and obtaining a detection result. Among them, the feature extraction network is used to extract features from the input face image data, the classification network is used to classify the features extracted by the feature extraction network, and the classification results of the classification network include the presence of a fake face in the face image and the absence of a fake face in the face image.

[0061] In one example, Figure 3 is a schematic diagram of a face detection model according to an exemplary embodiment. Figure 3 , the feature extraction network includes the sequentially connected input layer, initial convolution layer, deep convolution layer and average pooling layer, the initial convolution layer includes the convolution layer and the pooling layer, the deep convolution layer includes the convolution layer (multiple convolution kernels) and the pooling layer, and the classification network includes the sequentially connected average pooling layer, fully connected layer, activation function, fully connected layer and sigmoid activation function. Of course, Figure 3 This is only an example of a model structure of a face detection model. The face detection model may also have other structures, which is not limited in the embodiments of the present disclosure.

[0062] Before using the face detection model, it is necessary to first train the face detection model. The training process of the face detection model includes: obtaining sample data, the sample data includes a first sample video, a first label corresponding to the first sample video, a second sample video, and a second label corresponding to the second sample video. The first sample video is a video that does not contain a fake face. The first label is used to characterize that there is no fake face in the first sample video. The second sample video is a video that contains a fake face. The second label is used to characterize that there is a fake face in the second sample video; based on the sample data, training an initial face detection model to be trained to obtain a face detection model.

[0063] Since the face detection model needs to detect whether there is a fake face in the video, that is, the face detection model needs to be able to detect face images with fake faces and also detect face images without fake faces. Therefore, during the training process, it is necessary to use the first sample video without fake faces and the second sample video with fake faces for training together.

[0064] In some embodiments, the first label corresponding to the first sample audio data and the second label corresponding to the second sample audio data can be manually marked by the user, so as to improve the accuracy of the training data and thereby improve the accuracy of the trained face detection model.

[0065] Optionally, based on the sample data, an initial face detection model to be trained is trained to obtain a face detection model, including: based on the first sample video, obtaining first sample face image data, based on the second sample video, obtaining second sample face image data, respectively inputting the first sample face image data and the second sample face image data into the initial face detection model to obtain a first predicted detection result of the first sample face image data and a second predicted detection result of the second sample face image data, and based on the difference between the first predicted detection result and the first label, and the difference between the second predicted detection result and the second label, adjusting the model parameters of the initial face detection model to obtain the face detection model. The first sample face image data includes a face image extracted from a video frame included in the first sample video, and the second sample face image data includes a face image extracted from a video frame included in the second sample video. The implementation method of obtaining the sample face image data is similar to the implementation method of obtaining the face image data in the above embodiment, and will not be repeated here.

[0066] It should be noted that the difference between the first predicted detection result and the first label, and the difference between the second predicted detection result and the second label can characterize whether the output of the current initial face detection model is accurate, so that the model parameters of the initial face detection model can be adjusted based on the difference between the predicted detection result and the label. For example, when the first predicted detection result and the first label are different, and / or the second predicted detection result and the second label are different, it can be determined that the model parameters of the initial face detection model are not appropriate, and therefore, the model parameters of the initial face detection model need to be adjusted until the precision and recall of the trained initial face detection model meet the requirements, and the trained initial face detection model is used as the face detection model.

[0067] Optionally, the sample data is divided into training data and test data, and an initial face detection model is trained based on the training data to obtain a trained initial face detection model. Then, the precision and recall of the trained initial face detection model are tested based on the test data. When the precision and recall of the trained initial face detection model meet the requirements, the trained initial face detection model is used as the face detection model.

[0068] In one example, Figure 4It is a schematic diagram of a model training process according to an exemplary embodiment. First, a first sample video, a second sample video, a first label corresponding to the first sample video, and a second label corresponding to the second sample video are obtained. Then, for a first sample video frame in the first sample video, a first sample face image is extracted from the first sample video frame. Based on the extracted first sample face image, the first sample face image data is determined. For a second sample video frame in the second sample video, a second sample face image is extracted from the second sample video frame. Based on the extracted second sample face image, the second sample face image data is determined. Finally, based on the first sample face image data, the first label, the second sample face image data and the second label, the initial face detection model is trained and tested to obtain a face detection model. After that, the face detection model can be used for model prediction, that is, the face detection model is used to detect the input face image data.

[0069] In some embodiments, Figure 5 A schematic diagram of a video detection method is given. During the current video call, the terminal device background monitors the original video of the call in real time, obtains the video to be detected during the current call, and then calls the face detection model that has been trained offline to process the video to be detected to determine whether the face in the video to be detected is a fake face. If there is a fake face, a prompt message is fed back to the user. In addition, the offline training process of the face detection model can refer to Figure 4 In the process shown, for example, sample data can be obtained first, and then data processing can be performed on the sample data (i.e., the sample data can be divided into training data and test data), and then sample face image data can be obtained based on the sample data, and finally model training can be performed based on the sample face image data to obtain a face detection model.

[0070] In addition, it should be noted that the video detection method shown in the above embodiments may be a function set in the terminal device, and the user may choose to turn the function on or off based on demand; or, the function may be turned on by default.

[0071] Figure 6 is a block diagram of a video detection device according to an exemplary embodiment, which is configured in an electronic device, see Figure 6 , the device comprises: The first acquisition module 601 is configured to acquire a video to be detected during the process of displaying the original video on the terminal device; the video to be detected is a video intercepted from the original video; The second acquisition module 602 is configured to acquire face image data based on the video to be detected; the face image data includes a face image extracted from a video frame included in the video to be detected; The processing module 603 is configured to call the face detection model to detect the face image data and obtain the detection result of the video to be detected; the face detection model is used to detect whether the face in the face image is a fake face, and the detection result is used to characterize whether there is a fake face in the video to be detected.

[0072] In some embodiments, the second acquisition module 602 is configured to: For each video frame in the video to be detected, extract the facial key points in the video frame, and extract the facial image in the video frame based on the facial key points; Based on the extracted at least one face image, face image data is determined.

[0073] In some embodiments, the first acquisition module 601 is configured to: The original video is intercepted by using a preset sliding window to obtain a video to be detected; the preset sliding window is used to intercept a video of a preset length, and the video to be detected is a video clip of the preset length that is most recently displayed in the original video.

[0074] In some embodiments, the first acquisition module 601 is configured to: In response to an instruction to record an original video, start recording the original video, and when the recording time reaches a preset time, stop recording to obtain a video to be detected; or, A video segment of a preset length is copied from the original video, and the copied video segment is used as the video to be detected; wherein the original video is a video decoded by a preset decoder or a video in a buffer queue.

[0075] In some embodiments, the first acquisition module 601 is configured to: Based on a preset frame rate, a video segment of a preset length is copied from the original video; the preset frame rate is used to represent the number of video frames copied from the original video per second, and the preset frame rate is less than or equal to the video frame rate of the original video.

[0076] In some embodiments, the face detection model includes a feature extraction network and a classification network, and the processing module 603 is configured to: Calling a feature extraction network to extract features from the face image data to obtain target features corresponding to the face image data; the target features are used to characterize features of the face in the face image data; Call the classification network to classify the target features and obtain the detection results.

[0077] In some embodiments, the processing module 603 is further configured to: Obtaining sample data, the sample data including a first sample video, a first label corresponding to the first sample video, a second sample video, and a second label corresponding to the second sample video, wherein the first sample video is a video that does not contain a forged face, and the first label is used to indicate that there is no forged face in the first sample video, and the second sample video is a video that contains a forged face, and the second label is used to indicate that there is a forged face in the second sample video; Based on the sample data, an initial face detection model to be trained is trained to obtain a face detection model.

[0078] In some embodiments, the processing module 603 is further configured to: Based on the first sample video, first sample face image data is obtained; the first sample face image data includes a face image extracted from a video frame included in the first sample video; Based on the second sample video, obtaining second sample face image data; the second sample face image data includes a face image extracted from a video frame included in the second sample video; Inputting the first sample face image data and the second sample face image data into the initial face detection model respectively, to obtain a first predicted detection result of the first sample face image data and a second predicted detection result of the second sample face image data; Based on the difference between the first predicted detection result and the first label, and the difference between the second predicted detection result and the second label, the model parameters of the initial face detection model are adjusted to obtain the face detection model.

[0079] In some embodiments, the processing module 603 is further configured to: In response to the detection result indicating that a forged face exists in the video to be detected, a prompt message is issued, where the prompt message is used to prompt the target object that there is a risk of a forged face in the original video.

[0080] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0081] An embodiment of the present disclosure further provides an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the video detection method in the above embodiment.

[0082] Figure 7 is a block diagram of an electronic device 700 according to an exemplary embodiment.

[0083] Reference Figure 7, the electronic device 700 may include one or more of the following components: a processing component 702 , a memory 704 , a power component 706 , a multimedia component 708 , an audio component 710 , an input / output (I / O) interface 712 , a sensor component 714 , and a communication component 716 .

[0084] The processing component 702 generally controls the overall operation of the electronic device 700, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 702 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 702 may include one or more modules to facilitate the interaction between the processing component 702 and other components. For example, the processing component 702 may include a multimedia module to facilitate the interaction between the multimedia component 708 and the processing component 702.

[0085] The memory 704 is configured to store various types of data to support operations on the electronic device 700. Examples of such data include instructions for any application or method operating on the electronic device 700, contact data, phone book data, messages, pictures, videos, etc. The memory 704 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0086] The power supply component 706 provides power to various components of the electronic device 700. The power supply component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 700.

[0087] The multimedia component 708 includes a screen that provides an output interface between the electronic device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 708 includes a front camera and / or a rear camera. When the electronic device 700 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0088] The audio component 710 is configured to output and / or input audio signals. For example, the audio component 710 includes a microphone (MIC), and when the electronic device 700 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 704 or sent via the communication component 716. In some embodiments, the audio component 710 also includes a speaker for outputting audio signals.

[0089] I / O interface 712 provides an interface between processing component 702 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0090] The sensor assembly 714 includes one or more sensors for providing various aspects of status assessment for the electronic device 700. For example, the sensor assembly 714 can detect the open / closed state of the electronic device 700, the relative positioning of components, such as the display and keypad of the electronic device 700, and the sensor assembly 714 can also detect the position change of the electronic device 700 or a component of the electronic device 700, the presence or absence of user contact with the electronic device 700, the orientation or acceleration / deceleration of the electronic device 700, and the temperature change of the electronic device 700. The sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 714 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 714 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0091] The communication component 716 is configured to facilitate wired or wireless communication between the electronic device 700 and other devices. The electronic device 700 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 716 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 716 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0092] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0093] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 704 including instructions, and the instructions can be executed by a processor 820 of the electronic device 700 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0094] The embodiments of the present disclosure further provide a non-temporary computer-readable storage medium. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the video detection method in the above embodiments.

[0095] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0096] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A video detection method, characterized in that: The method comprises: In the process of displaying the original video on the terminal device, obtaining a video to be detected; the video to be detected is a video intercepted from the original video; Based on the video to be detected, acquiring face image data; the face image data includes a face image extracted from a video frame included in the video to be detected; A face detection model is called to detect the face image data to obtain a detection result of the video to be detected; the face detection model is used to detect whether the face in the face image is a forged face, and the detection result is used to characterize whether there is a forged face in the video to be detected.

2. The video detection method according to claim 1, characterized in that: The step of acquiring face image data based on the video to be detected includes: For each video frame in the video to be detected, extracting facial key points in the video frame, and extracting a facial image in the video frame based on the facial key points; The facial image data is determined based on the extracted at least one facial image.

3. The video detection method according to claim 1, characterized in that: The step of obtaining the video to be detected includes: The original video is intercepted using a preset sliding window to obtain the video to be detected; the preset sliding window is used to intercept a video of a preset length, and the video to be detected is a video clip of the preset length that is most recently displayed in the original video.

4. The video detection method according to claim 3, characterized in that: The method of intercepting the original video by using a preset sliding window to obtain the video to be detected includes: In response to a recording instruction for the original video, start recording the original video, and when the recording time reaches the preset time, stop recording to obtain the video to be detected; or, A video segment of the preset length is copied from the original video, and the copied video segment is used as the video to be detected; wherein the original video is a video decoded by a preset decoder or a video in a buffer queue.

5. The video detection method according to claim 4, characterized in that: The step of copying the video segment of the preset duration from the original video includes: Based on a preset frame rate, a video segment of the preset length is copied from the original video; the preset frame rate is used to represent the number of video frames copied from the original video per second, and the preset frame rate is less than or equal to the video frame rate of the original video.

6. The video detection method according to claim 1, characterized in that: The face detection model includes a feature extraction network and a classification network. The face detection model is called to detect the face image data to obtain the detection result of the video to be detected, including: Calling the feature extraction network to perform feature extraction on the face image data to obtain target features corresponding to the face image data; the target features are used to characterize features of the face in the face image data; The classification network is called to classify the target features to obtain the detection result.

7. The video detection method according to claim 1, characterized in that: The training process of the face detection model includes: Acquire sample data, the sample data including a first sample video, a first label corresponding to the first sample video, a second sample video, and a second label corresponding to the second sample video, the first sample video being a video that does not contain a forged face, the first label being used to indicate that there is no forged face in the first sample video, the second sample video being a video that contains a forged face, and the second label being used to indicate that there is a forged face in the second sample video; Based on the sample data, an initial face detection model to be trained is trained to obtain the face detection model.

8. The video detection method according to claim 7, characterized in that: The step of training an initial face detection model to be trained based on the sample data to obtain the face detection model comprises: Based on the first sample video, first sample face image data is obtained; the first sample face image data includes a face image extracted from a video frame included in the first sample video; Based on the second sample video, obtaining second sample face image data; the second sample face image data includes a face image extracted from a video frame included in the second sample video; Inputting the first sample face image data and the second sample face image data into the initial face detection model respectively, and obtaining a first predicted detection result of the first sample face image data and a second predicted detection result of the second sample face image data; Based on the difference between the first predicted detection result and the first label, and the difference between the second predicted detection result and the second label, the model parameters of the initial face detection model are adjusted to obtain the face detection model.

9. The video detection method according to claim 1, characterized in that: The method further comprises: In response to the detection result indicating that a forged face exists in the video to be detected, a prompt message is issued, where the prompt message is used to prompt a target object that there is a risk of a forged face in the original video.

10. A video detection device, characterized in that: The device comprises: A first acquisition module is configured to acquire a video to be detected during the process of displaying an original video on a terminal device; the video to be detected is a video intercepted from the original video; A second acquisition module is configured to acquire face image data based on the video to be detected; the face image data includes a face image extracted from a video frame included in the video to be detected; The processing module is configured to call a face detection model to detect the face image data to obtain a detection result of the video to be detected; the face detection model is used to detect whether the face in the face image is a forged face, and the detection result is used to characterize whether there is a forged face in the video to be detected.

11. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the video detection method as described in any one of claims 1-9.

12. A non-transitory computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the video detection method as described in any one of claims 1 to 9.

Images