Video detection method and device

Abstract

The invention discloses a video detection method and device, and the method comprises the steps: obtaining an environment image sequence based on a video frame sequence corresponding to a target video; target sensor data corresponding to the target video is obtained, the target sensor data is provided by a target sensor configured in target equipment for collecting the target video, and the target sensor is used for sensing motion of the target equipment; and determining whether the target video is a false video according to the environment image sequence and the target sensor data.

Description

Technical Field

[0001] The embodiments of the present specification belong to the field of computer technology, and more particularly, to a video detection method and device. Background Art

[0002] Deepfake is a simulation technology that uses deep learning algorithms to synthesize video or audio. It can forge facial features or imitate the voice of a person in the video or audio. With the rapid development of Deepfake technology, it has become easier and more realistic to produce fake videos, posing a serious threat to personal privacy, information authenticity and social trust.

[0003] It is hoped that there will be a new technical solution that can more accurately detect whether the relevant video is a fake video. Summary of the invention

[0004] The object of the present invention is to provide a video detection method and device.

[0005] In a first aspect, a video detection method is provided, comprising: obtaining an environmental image sequence based on a video frame sequence corresponding to a target video; acquiring target sensor data corresponding to the target video, the target sensor data being provided by a target sensor configured in a target device that collects the target video, the target sensor being used to sense the movement of the target device; and determining whether the target video is a false video based on the environmental image sequence and the target sensor data.

[0006] In a second aspect, a video detection device is provided, which includes a video processing unit configured to obtain an environmental image sequence based on a video frame sequence corresponding to a target video; a data acquisition unit configured to obtain target sensor data corresponding to the target video, the target sensor data being provided by a target sensor configured in a target device that collects the target video, the target sensor being used to sense the movement of the target device; and a detection processing unit configured to determine whether the target video is a false video based on the environmental image sequence and the target sensor data.

[0007] In a third aspect, a computing device is provided, comprising a memory and a processor, wherein the memory stores a computer program / instructions, and when the processor executes the computer program / instructions, the method described in the first aspect is implemented.

[0008] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program / instruction is stored. When the computer program / instruction is executed in a computing device, the computing device executes the method described in the first aspect.

[0009] In the technical solution provided by the embodiments of this specification, the environmental image sequence obtained based on the video frame sequence corresponding to the target video can reflect the movement of the target device in the process of collecting the target video; the target sensor configured in the target device and capable of sensing the movement of the target device, and the target sensor data collected in the process of the target device collecting the target video, can also reflect the movement of the target device in the process of collecting the target video; when the target video is a real video collected by the target device and not a false video that has been subjected to an injection attack, the movement of the target device reflected by the environmental image sequence should be consistent with the movement of the target device reflected by the target sensor data, and therefore, it can be more accurately determined whether the target video is a false video based on the environmental image sequence and the target sensor data. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0011] Figure 1 A schematic diagram of a technical scenario of the technical solution provided in the embodiments of this specification;

[0012] Figure 2 A flowchart of a video detection method provided in an embodiment of this specification;

[0013] Figure 3 A schematic diagram of data flow in a video detection process provided in an embodiment of this specification;

[0014] Figure 4 The diagram is a schematic diagram of the structure of a video detection device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0015] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0016] With the rapid development of Deepfake technology, the production of fake videos has become easier and more realistic, posing a serious threat to personal privacy, information authenticity and social trust. Intruders may use attack methods including Deepfake technology to maliciously modify real videos to obtain fake videos, or even directly forge fake videos without real videos.

[0017] In technical scenarios involving video acquisition, such as eKYC (Electronic Know Your Customer), face recognition, video calls, iris recognition, etc., it is necessary to be able to effectively identify whether the video from the relevant device is a fake video in order to resist the identity authenticity threats brought by various attack methods including Deepfake technology.

[0018] The inventors have found that a target device equipped with an image acquisition unit is usually equipped with a target sensor to sense the movement of the target device. When the target device acquires video through the image acquisition unit, the positional relationship between the image acquisition unit and the target sensor usually remains unchanged. Therefore, in the real video acquired by the target device that has not been maliciously modified, the change of the environment is correlated with the movement of the target device; however, in a fake video, this correlation is often destroyed.

[0019] In view of this, at least one video detection method, apparatus, computing device and computer-readable storage medium are provided in the embodiments of the present specification. The environmental image sequence obtained based on the video frame sequence corresponding to the target video can reflect the movement of the target device in the process of collecting the target video; the target sensor configured in the target device and capable of sensing the movement of the target device, and the target sensor data collected in the process of the target device collecting the target video can also reflect the movement of the target device in the process of collecting the target video; when the target video is a real video collected by the target device and not a false video that has been subjected to an injection attack, the movement of the target device reflected by the environmental image sequence should be consistent with the movement of the target device reflected by the target sensor data. Therefore, it is possible to more accurately determine whether the target video is a false video based on the environmental image sequence and the target sensor data.

[0020] Figure 1 A schematic diagram of a technical scenario of the technical solution provided in the embodiments of this specification.

[0021] Reference Figure 1As shown, in the target device equipped with the image acquisition unit, a target sensor is also configured to sense the movement of the target device. The target sensor refers to a spatial attitude measurement unit in the target device, which may include but is not limited to one or more of various sensors such as an accelerometer, a gyroscope, and a magnetometer. For example, the target sensor may be an inertial measurement unit (IMU), which integrates 3 single-axis accelerometers and 3 single-axis gyroscopes, and may even integrate a magnetometer for assisting in determining the attitude of the target device itself. The aforementioned target devices may include, for example, but are not limited to mobile phones, tablet computers, and personal computers (PCs).

[0022] For various technical scenarios involving video acquisition, when the target device acquires the target video through its own configured image acquisition unit, the target sensor configured in the target device can also simultaneously acquire the target sensor data corresponding to the target video. When the target video needs to be sent to a relevant recipient, such as a server of a service platform, the target sensor data can also be sent to the recipient of the target video so that the recipient can detect whether the target video is a false video.

[0023] Figure 2 This is a flow chart of a video detection method provided in an embodiment of this specification. The method can be executed by any device, platform, equipment or equipment cluster with computing or processing capabilities, for example, by a server of a related service platform.

[0024] Reference Figure 2 As shown, the method may include but is not limited to part or all of the following steps S201 to S205.

[0025] Step S201: obtaining an environment image sequence based on a video frame sequence corresponding to a target video.

[0026] In a possible implementation, the environment image sequence may be an unprocessed video frame sequence.

[0027] Depending on the various specific matters that need to be implemented using the target video in the subsequent process, such as the need to use the target video to perform biometric identification of the target user, such as face recognition or iris recognition, the target video may contain a person image area. During the process of the relevant target device collecting the target video through the image acquisition unit configured therein, if the person within the field of view of the image acquisition unit is not in a static state, the person image area is difficult to reflect the movement of the target device when collecting the target video, and it will also bring a lot of resource overhead to the subsequent process of predicting the movement of the target device.

[0028] In a possible implementation, target detection and portrait segmentation techniques may be used to process a video frame sequence to eliminate the human image region in the video frame, thereby obtaining an environmental image sequence corresponding to the video frame sequence, so that the movement of the target device can be predicted more efficiently and accurately based on the environmental image sequence. Exemplarily, each video frame in the video frame sequence may be first processed by various target detection models including Grounding Dino to identify and mark the human face image region in each video frame; then, in each video frame, the pixel value of each pixel point in the human face image region is set to a preset value, such as 0, thereby obtaining an environmental image sequence corresponding to the video frame sequence.

[0029] Step S203, acquiring target sensor data corresponding to the target video, wherein the target sensor data is provided by a target sensor configured in a target device that captures the target video, and the target sensor is used to sense the movement of the target device.

[0030] The target sensor data refers to the sensor data collected by the target sensor configured by the target device during the process of collecting the target video through the image acquisition unit configured by the target device. Exemplarily, the video frame sequence corresponding to the target video includes N video frames arranged in sequence, where the collection time of the first video frame is time t1, the collection time of the Nth video frame is time t N , the target sensor data can be the target sensor data from time t1 to t N The sensor data is collected continuously.

[0031] As mentioned above, the target sensor refers to a spatial attitude measurement unit in the target device, which may include but is not limited to one or more of various sensors such as an accelerometer, a gyroscope, and a magnetometer. For example, the target sensor may be an inertial measurement unit, which integrates three single-axis accelerometers and three single-axis gyroscopes, and may even integrate a magnetometer.

[0032] Step S205: determining whether the target video is a false video according to the environment image sequence and the target sensor data.

[0033] In a possible implementation, the first pose change information corresponding to the target device can be predicted based on the environmental image sequence; the second pose change information corresponding to the target device can be predicted based on the target sensor data; it is determined whether the first pose change information matches the second pose change information; if not, it is determined that the target video is a false video.

[0034] The following first describes, by way of example, a process of predicting the first position change information corresponding to the target device.

[0035] Various possible methods such as feature point method, direct method or training machine learning model can be used to process the environmental image sequence to obtain the first pose change information of the target device. Taking the feature point method as an example, it is first assumed that the environmental image sequence includes N environmental images arranged in sequence, and the acquisition time of the video frame corresponding to any i-th environmental image acquired by the target device is recorded as t i , then: first, feature detection algorithms such as ORB (Oriented FAST and RotatedBRIEF), scale-invariant feature transform (SIFT) or speeded up robust features (SURF) can be used to extract feature points from N environmental images. Then, Brute-Force Matcher or fast approximate nearest neighbor search based on FLANN (Fast Library for Approximate Nearest Neighbors) can be used to match multiple feature point pairs between the i-th environmental image and the i-1-th environmental image. Then, the RANSAC (random sample consensus) algorithm or other possible screening algorithms can be used to eliminate feature point pairs that may be abnormal. Then, based on the filtered feature point pairs, the essential matrix corresponding to the i-th environmental image and the i-1-th environmental image is calculated, and the third relative position and rotation of the i-th environmental image relative to the i-1-th environmental image is obtained by decomposing the essential matrix. 1i , that is, to obtain the time t i The corresponding third relative posture S 1i Among them, the third relative posture S of the above example 1i The rotation information R can be included 1i and displacement vector T 1i ; Rotation information R 1i Used to indicate the target device at time t i Relative to the target device at time t i-1 The rotation angle in the x, y, and z axes of the three-dimensional space; the displacement vector T 1i Used to indicate the target device at time t i Relative to the target device at time t i-1 It should be noted that when the value of i is 1, the values ​​of the first relative position and the third relative position of the first environment image relative to the non-existent zeroth environment image can be recorded as 0.

[0036] Through the above example process, the time t1~t N After the third relative postures corresponding to each of the N moments are calculated, the first posture change information can be determined. N The third relative posture corresponding to each of the N moments is determined as the first posture change information. N The rotation information included in the third relative posture corresponding to each of the N moments is determined as the first posture change information. N The displacement vector included in the third relative posture corresponding to each of the N moments is determined as the first posture change information. N The third relative postures corresponding to the N moments are calculated, and the N first relative postures of the target device when collecting the 1st to Nth video frames are calculated relative to the target device when collecting the 1st video frame, that is, the N first relative postures of the target device at time t1~t N N first relative postures of the target device at time t1 at the same time, each of which undergoes a posture change; for any i-th environment image, any i-th video frame or time t i , the corresponding i-th first relative position and posture may include: when the target device collects the video frame corresponding to the i-th environment image (i.e., time t i ), the rotation angle of the target device in the x, y, and z axes of the three-dimensional space relative to the time when the target device captures the video frame corresponding to the first environmental image (i.e., time t1); and / or, the target device at time t i The distance that the target device is displaced on the x, y, and z axes in the three-dimensional space at time t1. N The first relative postures corresponding to the N moments are determined as the first posture change information corresponding to the target device in the process of capturing the target video.

[0037] The following describes the process of predicting the second posture change information corresponding to the target device.

[0038] Take the target sensor including an accelerometer and a gyroscope as an example; the accelerometer can measure the linear acceleration of the target device on the x, y, and z axes in three-dimensional space, while the gyroscope can measure the rotation speed of the target device rotating in the x, y, and z axes. By integrating the data measured by the accelerometer and the gyroscope in the target sensor data, the position change of the target device relative to the initial state (the position and posture of the target device when collecting the first video frame) when collecting N video frames can be estimated, so that the second position change information corresponding to the target device in the process of collecting the target video can be determined based on the estimated position and posture changes.

[0039] Continuing with the previous example, let the acquisition time of the i-th video frame be recorded as time t i , the acquisition time of the i-1th video frame is recorded as time t i-1 , then according to the target sensor data, the acquisition time is at time t i-1 ~t i The sensor data between the two time periods is used to estimate the target device at time t i Relative to the target device at time t i-1 The fourth relative posture S of the i-th position change 2i , that is, calculate the time t i The corresponding fourth relative posture S 2i Among them, the fourth relative posture S of the above example 2i May include: rotation information R 2i and / or displacement vector T 2i ; Rotation information R 2i Used to indicate the target device at time t i Relative to the target device at time t i-1 The rotation angle in the x, y, and z axes of the three-dimensional space; the displacement vector T 2i Used to indicate the target device at time t i Relative to the target device at time t i-1 The distance of displacement on the x, y, and z axes in three-dimensional space.

[0040] Through the above example process, the time t1~t N After the fourth relative posture corresponding to each other, the above exemplary description based on the time t1 to t N The process of determining the first relative posture change information of each of the N moments corresponding to the third relative posture is similar, and can be based on the time t1~t N The fourth relative postures corresponding to the N moments are used to determine the second posture change information corresponding to the target device.

[0041] Various possible matching algorithms can be used to match the predicted first pose change information and the second pose change information. For example, the first pose change information and the second pose change information can be converted into a first embedding vector and a second embedding vector, respectively, and then the first similarity of the first embedding vector and the second embedding vector is calculated. When the first similarity is less than the first preset threshold, it indicates that the first pose change information and the second pose change information do not match. For another example, when the first pose change information includes N first relative poses and the second pose change information includes N second relative poses, the i-th similarity between the i-th first relative pose and the i-th second relative pose can be calculated, and then the first similarity of the first pose change information and the second pose change information is calculated based on the calculated N similarities. When the first similarity is less than the first preset threshold, it is determined that the first pose change information and the second pose change information do not match.

[0042] In one possible implementation, a first encoder can be used to process an environmental image sequence to obtain an environmental change representation vector; a second encoder can be used to process target sensor data to obtain a posture change representation vector, and the first encoder and the second encoder are machine learning models obtained through multi-modal contrast learning training; a second similarity between the environmental change representation vector and the posture change representation vector is calculated; when the second similarity is less than a second preset threshold, the target video is determined to be a false video.

[0043] Multimodal contrastive learning is a self-supervised learning method that aims to learn feature representations with special significance by comparing samples of different modalities in the data space. The core idea is to minimize the value of the loss function as the optimization goal, and update the parameters of the first encoder and the second encoder through one or more rounds to maximize the similarity between samples with associated relationships (or reduce the distance between them), while minimizing the similarity between samples without associated relationships (or increase the distance between them). The loss function used in the training process can usually be defined as minimizing the negative log-likelihood or maximizing the cross entropy loss function. Based on this, the first encoder and the second encoder can be trained based on the first sample pair (positive sample) and the second sample pair (negative sample); wherein the first sample pair includes a first environmental image sequence and a first sensor data, the first environmental image sequence is obtained based on a first video, and the first video and the first sensor data are collected by the same device in the same time period; and the second sample pair includes a second environmental image sequence and a second sensor data that are not associated.

[0044] It should be particularly noted that the absence of an association relationship between the second environment image sequence and the second sensor data may include: the second sensor data and the second video corresponding to the second environment image are not collected by the same device; or the second sensor data and the second video corresponding to the second environment image are collected by the same device at different times.

[0045] The first sensor data in the first sample pair can also be replaced by the posture change information of the relevant device predicted based on the first sensor data; similarly, the second sensor data in the second sample pair can also be replaced by the posture change information of the relevant device predicted based on the second sensor data. In this case, refer to Figure 3 As shown, in the aforementioned embodiment, a second encoder can also be used to process the second posture change information to obtain a posture change representation vector.

[0046] At the same time, combined with the various exemplary embodiments described above, when it is determined that the first posture change information and the second posture change information match, and the environment change representation vector and the posture change representation vector are similar, it can be determined that the detected target video is not a false video, that is, the target video is a real video captured by the relevant target device.

[0047] Based on the same concept as the above method embodiment, this specification embodiment also provides a video detection device 400. Figure 4 As shown, the device 400 includes: a video processing unit 401, configured to obtain an environmental image sequence based on a video frame sequence corresponding to a target video; a data acquisition unit 403, configured to obtain target sensor data corresponding to the target video, the target sensor data being provided by a target sensor configured in a target device that collects the target video, and the target sensor being used to sense the movement of the target device; a detection processing unit 405, configured to determine whether the target video is a false video based on the environmental image sequence and the target sensor data.

[0048] A computer-readable storage medium is also provided in an embodiment of the present specification, on which a computer program / instruction is stored. When the computer program / instruction is executed in a computer, the computer executes the video detection method provided in the aforementioned embodiments.

[0049] A computing device is also provided in an embodiment of the present specification, including a memory and a processor, wherein the memory stores a computer program / instruction, and when the processor executes the computer program / instruction, the video detection method provided in the aforementioned embodiments is implemented.

[0050] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0051] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0052] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a server system. Of course, the present application does not exclude that with the development of computer technology in the future, the computer that implements the functions of the above embodiments may be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0053] Although one or more embodiments of the present specification provide method operation steps as described in the embodiments or flow charts, more or less operation steps may be included based on conventional or non-creative means. The order of steps listed in the embodiments is only one way of executing the order of many steps, and does not represent the only execution order. When the device or terminal product in practice is executed, it can be executed in sequence or in parallel according to the method shown in the embodiments or the drawings (for example, a parallel processor or a multi-threaded processing environment, or even a distributed data processing environment). The term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, product or equipment including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, product or equipment. In the absence of more restrictions, it is not excluded that there are other identical or equivalent elements in the process, method, product or equipment including the elements. For example, if the words first, second, etc. are used to represent the name, they do not represent any specific order.

[0054] For the convenience of description, the above devices are described in various modules according to their functions. Of course, when implementing one or more of the present specification, the functions of each module can be implemented in the same or more software and / or hardware, or the module implementing the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0055] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0056] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0057] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0058] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0059] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0060] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage, graphene storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0061] It should be understood by those skilled in the art that one or more embodiments of the present specification may be provided as a method, system or computer program product. Therefore, one or more embodiments of the present specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, one or more embodiments of the present specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0062] One or more embodiments of the present specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0063] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. In the description of this specification, the description of the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, in the absence of mutual contradiction, a person skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0064] The above description is only an example of one or more embodiments of the present specification and is not intended to limit one or more embodiments of the present specification. For those skilled in the art, one or more embodiments of the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims.

Claims

1. A video detection method, the method comprising: Based on the video frame sequence corresponding to the target video, an environment image sequence is obtained; Acquire target sensor data corresponding to the target video, where the target sensor data is provided by a target sensor configured in a target device that acquires the target video, and the target sensor is used to sense movement of the target device; Determine whether the target video is a false video according to the environment image sequence and the target sensor data.

2. According to the method of claim 1, obtaining the environment image sequence based on the video frame sequence corresponding to the target video comprises: For the video frame sequence corresponding to the target video, the character image area in the video frame is eliminated to obtain the environment image sequence.

3. The method according to claim 1, wherein determining whether the target video is a false video based on the environment image sequence and the target sensor data comprises: Predicting first position change information corresponding to the target device according to the environment image sequence; Predicting second posture change information corresponding to the target device according to the target sensor data; Determining whether the first posture change information matches the second posture change information; If not, it is determined that the target video is a false video.

4. The method according to claim 3, wherein the first posture change information includes an i-th first relative posture of the target device that undergoes a posture change when the target device captures any i-th video frame relative to the target device when the target device captures the first video frame; the second posture change information includes an i-th second relative posture of the target device that undergoes a posture change when the target device captures any i-th video frame relative to the target device when the target device captures the first video frame; in, The determining whether the first posture change information and the second posture change information match includes: calculating the i-th similarity between the i-th first relative posture and the i-th second relative posture, and calculating the first similarity between the first posture change information and the second posture change information based on the i-th similarity, and when the first similarity is less than a first preset threshold, determining that the first posture change information and the second posture change information do not match.

5. The method according to claim 1, wherein determining whether the target video is a false video based on the environment image sequence and the target sensor data comprises: Processing the environment image sequence using a first encoder to obtain an environment change representation vector; Processing the target sensor data using a second encoder to obtain a posture change representation vector, wherein the first encoder and the second encoder are machine learning models obtained through multi-modal contrastive learning training; Calculating a second similarity between the environment change representation vector and the posture change representation vector; When the second similarity is less than a second preset threshold, it is determined that the target video is a false video.

6. According to the method of claim 5, the first encoder and the second encoder are trained based on a first sample pair and a second sample pair; the first sample pair includes a first environmental image sequence and first sensor data, the first environmental image sequence is obtained based on a first video, and the first video and the first sensor data are collected by the same device in the same time period; the second sample pair includes a second environmental image sequence and a second sensor data that have no association relationship.

7. The method according to claim 6, wherein the second environment image is obtained based on the second video; The second video and the second sensor data are collected by different devices, or the second video and the second sensor data are collected by the same device in different time periods.

8. The method according to claim 1, wherein the target video is used to support biometric identification of the target user. 9 . The method according to claim 1 , wherein the target sensor is at least one of the following sensors: an accelerometer, a gyroscope, and a magnetometer.

10. A video detection device, comprising: A video processing unit configured to obtain an environment image sequence based on a video frame sequence corresponding to a target video; a data acquisition unit configured to acquire target sensor data corresponding to the target video, wherein the target sensor data is provided by a target sensor configured in a target device that acquires the target video, and the target sensor is used to sense the movement of the target device; The detection processing unit is configured to determine whether the target video is a false video based on the environment image sequence and the target sensor data.

11. A computing device, comprising a memory and a processor, wherein the memory stores a computer program / instruction, and when the processor executes the computer program / instruction, the method according to any one of claims 1 to 9 is implemented.

12. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed in a computing device, the computing device executes the method according to any one of claims 1 to 9.

Images