Video authentic identification and time positioning method and device, electronic equipment and storage medium

Abstract

The invention discloses a video authentic identification and time positioning method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: carrying out the feature extraction of a video and an audio in a video file, and obtaining the audio and video features with the aligned time dimensions; performing cross-modal cross reconstruction on the extracted audio and video features, and supervising the step of cross-modal cross reconstruction by using the spatial difference between true and false features so as to obtain spatial error features; according to each feature, determining the tampering probability of the corresponding current frame, and then respectively carrying out frame classification on each current frame; and performing time positioning on the tampered data frame in the video file according to each feature and the tampered probability of the corresponding current frame. According to the method, the error between the audio and video modals is obtained in a mode of monitoring the reconstruction process through cross-modal cross reconstruction and utilizing the spatial difference between true and false features, so that tampered video frames are more accurately concerned, and the accuracy of authentic identification and time positioning can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a video authentication and time positioning method, device, electronic device and storage medium. Background Art

[0002] With the development of deep tampering technology, the videos it generates are becoming more realistic and difficult to distinguish with the naked eye. Existing technologies often only target single-mode tampering, resulting in a limited identification range. Even if some existing technologies use multi-modal information, they are often limited to the binary classification of tampering or not, and cannot determine the specific tampered segments; some technical solutions that use multi-modality to locate the time of deep tampering have low identification and positioning accuracy. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a video authentication and time positioning method, device, electronic device and storage medium to improve the accuracy of video authentication and time positioning.

[0004] To achieve the above purpose, an embodiment of the present application provides a video authentication and time positioning method, which includes the following steps:

[0005] Extract features of the video and audio in the video file respectively to obtain input video features and input audio features aligned in the time dimension;

[0006] Performing cross-modal cross-reconstruction on the input video features and the input audio features, and using the spatial difference between the true and false features to supervise the cross-modal cross-reconstruction step, thereby obtaining a spatial error feature;

[0007] Determine the probability that the corresponding current frame is tampered with according to the input video feature, the input audio feature and the spatial error feature, and then classify each of the current frames respectively;

[0008] The tampered data frame in the video file is temporally located based on the input video feature, the input audio feature, the spatial error feature and the probability that the corresponding current frame has been tampered with; wherein the data frame includes at least one of the current frame corresponding to the input video feature or the current frame corresponding to the input audio feature.

[0009] In some embodiments, before extracting features from the video and audio in the video file to obtain input video features and input audio features aligned in the time dimension, the method further includes a step of preprocessing the video file, and the step of preprocessing the video file includes the following steps:

[0010] Extract the video and the audio from the video file;

[0011] Filling the video and the audio respectively;

[0012] Scaling the size of the padded video and performing channel adjustment to obtain a video tensor of a first shape;

[0013] Extracting a Mel-spectrogram from the padded audio, and then representing the Mel-spectrogram as an audio tensor of a second shape;

[0014] The step of extracting features of the video and audio in the video file respectively to obtain input video features and input audio features aligned in the time dimension comprises the following steps:

[0015] Feature extraction is performed on the video tensor and the audio tensor respectively to obtain input video features and input audio features aligned in time dimension.

[0016] In some embodiments, before the step of performing cross-modal cross-reconstruction on the input video features and the input audio features and supervising the cross-modal cross-reconstruction using the spatial difference between the true and false features to obtain the spatial error features, the method further includes the step of training a cross-modal cross-reconstruction module, and the step of training the cross-modal cross-reconstruction module includes the following steps:

[0017] The V2A network is trained using video features labeled with real and tampered video features; the A2V network is trained using audio features labeled with real and tampered audio features; wherein the cross-modal cross-reconstruction module includes the V2A network and the A2V network; the A2V network and the V2A network are both implemented using a 4-layer standard transformer;

[0018] The expression of the loss function of the cross-modal cross-reconstruction module includes:

[0019]

[0020] P a =V2A(IN v );

[0021] P v =A2V(IN a );

[0022] Where N represents the number of video frames, P vi Indicates the features of the i-th video frame predicted using the input audio features, IN vi Represents the input video features; P ai Indicates the features of the i-th audio frame predicted using the input video features, IN ai Represents the audio features of the input;

[0023] The expression of the reconstruction supervision function of the cross-modal cross-reconstruction module includes:

[0024]

[0025] Mask pred_a =P a -Real a ;

[0026]

[0027] Mask ref_a =Fake a -Real a =IN a -Real a ;

[0028] Mask ref_v =Fake v -Real v =IN v -Real v ;

[0029] P a =V2A(IN v );

[0030] P v =A2V(IN a );

[0031] Among them, Mask pred_v Mask represents the difference between the predicted video features and the reference real video features, which are reconstructed by inputting the tampered audio features to obtain the predicted video features. ref Mask represents the difference between the input tampered video features and the reference real video features; pred_a Mask represents the difference between the predicted audio features and the reference real audio features, which are reconstructed by inputting the tampered video features; ref_a represents the difference between the input tampered audio feature and the reference real audio feature; N represents the number of video frames, P v Fake represents the video features predicted using the input audio features. v =IN v Represents the input tampered video feature; P a Indicates the audio features predicted using the input video features, Fake a =IN a Represents the tampered audio feature of the input; Real aIndicates the corresponding real audio features, Real v Indicates the corresponding real video features.

[0032] In some embodiments, the step of performing cross-modal cross-reconstruction on the input video features and the input audio features and supervising the cross-modal cross-reconstruction using the spatial difference between the true and false features to obtain the spatial error features comprises the following steps:

[0033] Input the input video features into the V2A network, and use the V2A network to infer and obtain reconstructed audio features; input the input audio features into the A2V network, and use the A2V network to infer and obtain reconstructed video features;

[0034] Subtracting the input video feature from the reconstructed video feature to obtain a reconstructed video error feature; subtracting the input audio feature from the reconstructed audio feature to obtain a reconstructed audio error feature;

[0035] The reconstructed video error feature and the reconstructed audio error feature are input into a linear layer for linear weighting to obtain the spatial error feature.

[0036] In some embodiments, the method further comprises the following steps:

[0037] When the training converges, the expected distribution of the reconstructed features is determined as follows:

[0038]

[0039] Among them, P a,i represents the features of the audio predicted using the input video features; v,i Fake represents the features of the video predicted using the input audio features; a,i Fake represents the features of the corresponding tampered audio frames in the dataset used for training. v,i Represents the features of the corresponding tampered video frames in the dataset used for training;

[0040] Then, when the training converges, the distribution of the spatial error characteristics output after the difference is determined as follows:

[0041]

[0042] Among them, Out a,i The audio feature output by the cross-modal cross-reconstruction module is used as the first spatial error; Out v,i The video features output by the cross-modal cross-reconstruction module are used as the second spatial error; the first spatial error and the second spatial error are used for subsequent audio and video frame classification and time positioning tasks;

[0043] The step of classifying each current frame separately comprises the following steps:

[0044] Performing frame classification on each of the current frames according to the loss function of frame classification;

[0045] The loss function of the frame classification includes:

[0046]

[0047] Among them, Lossf a 、Lossf v 、Lossf s Respectively represent the loss functions for frame classification using the input audio feature, the input video feature and the spatial error feature; is the standard cross entropy loss function.

[0048] In some embodiments, the temporal positioning of the tampered data frame in the video file according to the input video feature, the input audio feature, the spatial error feature and the probability that the current frame has been tampered with comprises the following steps:

[0049] Using a time positioning loss function, temporally positioning the tampered data frame in the video file according to the input video feature, the input audio feature, the spatial error feature, and the probability that the current frame has been tampered with;

[0050] The loss function of the time positioning is:

[0051]

[0052] Among them, Lossb a As the boundary matching loss functions for audio tampering time positioning, video tampering time positioning and any modality tampering time positioning respectively; N is the number of video frames; D is the maximum number of frames of the tampered data frame segment; represents the confidence of the tampered data frame segment starting from the i-th frame and ending at the i+j-th frame, Y b a,i j represents the corresponding true value.

[0053] In some embodiments, the method further comprises the following steps:

[0054] The tampered data frame and the corresponding time location are visually displayed.

[0055] To achieve the above-mentioned purpose, another aspect of the embodiment of the present application provides a video authentication and time positioning device, the device comprising:

[0056] A feature extraction unit, used to extract features of the video and audio in the video file respectively, to obtain input video features and input audio features aligned in time dimension;

[0057] A feature reconstruction unit, configured to perform cross-modal cross-reconstruction on the input video feature and the input audio feature, and supervise the cross-modal cross-reconstruction step by using the spatial difference between the true and false features, thereby obtaining a spatial error feature;

[0058] A feature classification unit, used to determine the probability that the corresponding current frame is tampered with according to the input video feature, the input audio feature and the spatial error feature, and then classify each of the current frames respectively;

[0059] A time positioning unit is used to temporally locate the tampered data frame in the video file based on the input video feature, the input audio feature, the spatial error feature and the probability that the corresponding current frame has been tampered with; wherein the data frame includes at least one of the current frame corresponding to the input video feature or the current frame corresponding to the input audio feature.

[0060] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned video authentication and time positioning method when executing the computer program.

[0061] To achieve the above objectives, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned video authentication and time positioning method is implemented.

[0062] The embodiments of the present application include at least the following beneficial effects:

[0063] The present application can extract features from the video and audio in the video file respectively, and obtain input video features and input audio features aligned in the time dimension; perform cross-modal cross-reconstruction on the input video features and input audio features, and use the spatial difference between true and false features to supervise the cross-modal cross-reconstruction step, and then obtain spatial error features; determine the probability that the corresponding current frame has been tampered with based on the input video features, input audio features and spatial error features, and then perform frame classification on each current frame respectively; perform time positioning on the tampered data frame in the video file based on the input video features, input audio features, spatial error features and the probability that the corresponding current frame has been tampered with; wherein the data frame includes at least one of the current frame corresponding to the input video features or the current frame corresponding to the input audio features. The present application obtains the error between audio and video modalities by cross-modal cross-reconstruction, and then more accurately focuses on the specific tampered video frame, which can improve the accuracy of authentication and time positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0065] Figure 1 A schematic diagram of the process of the video authentication and time positioning method provided in the embodiment of the present application;

[0066] Figure 2 An example flow chart of step S100 provided in an embodiment of the present application;

[0067] Figure 3 An example flow chart of step S200 provided in an embodiment of the present application;

[0068] Figure 4 An example flow chart of step S120 provided in an embodiment of the present application;

[0069] Figure 5 An example flow chart of step S130 provided in an embodiment of the present application;

[0070] Figure 6 An example flow chart of a video authentication and time positioning method provided in an embodiment of the present application;

[0071] Figure 7 An example flow chart of training a cross-modal cross-reconstruction module provided in an embodiment of the present application;

[0072] Figure 8An example flowchart of reasoning using a cross-modal cross-reconstruction module provided in an embodiment of the present application;

[0073] Fig. 9 A schematic diagram of the structure of a video authentication and time positioning device provided in an embodiment of the present application;

[0074] Fig.10 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0076] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".

[0077] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0078] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0079] Before describing the embodiments of the present application in detail, some nouns and terms involved in the embodiments of the present application are first described. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations:

[0080] Deepfake: refers to the use of deep learning technology to generate false audio and video content using neural networks. This technology usually uses methods such as generative adversarial networks (GANs) to synthesize a person's facial expressions, voice or movements into another person's audio and video, thereby creating content that looks real but is actually tampered with.

[0081] Cross-Modal Reconstruction: refers to the process of reconstructing one modality (such as audio) through another modality (such as video). This technology is used in multimodal authentication to detect information inconsistencies between different modalities and help identify tampered content.

[0082] Temporal Localization: In video processing and forgery detection, temporal localization refers to accurately determining the time point in the video where the tampered content occurs. By analyzing the cross-modal reconstruction errors of video and audio, the time annotation and localization of the tampered clips can be achieved.

[0083] Mel-Spectrogram: Mel-Spectrogram is a technique for converting audio signals into spectral representations. It divides frequencies according to the Mel scale based on the perceptual characteristics of the human auditory system. The Mel-Spectrogram first converts the audio signal in the time domain into a frequency domain representation through a short-time Fourier transform (STFT) to obtain the amplitudes of different frequencies, and then maps these frequencies to the Mel scale. The result is a two-dimensional matrix, where the horizontal axis represents time, the vertical axis represents the frequency components after Mel conversion, and the color or intensity represents the energy of different frequencies in each time segment.

[0084] Deepfake video authentication and time positioning is one of the key technologies required for network security and forensic identification. It is a technology used to distinguish videos generated by Deepfake technology from real videos and locate specific tampered segments.

[0085] Deepfake videos can be tampered with in the visual modality, including but not limited to the traditional generative adversarial network GAN, the current popular diffusion model Diffusion Model, etc.

[0086] Deepfake videos can be tampered with in the auditory modality, including but not limited to voice cloning, text-to-speech (TTS), etc.

[0087] Deepfake videos can also be tampered with in both visual and auditory modalities, for example using lip-sync technology to make the video image have consistent lip movements with the tampered audio.

[0088] With the development of Deepfake, the videos it generates are becoming more and more realistic and difficult to distinguish with the naked eye. Existing technologies often only target single-modal tampering, while ignoring the auxiliary role that the corresponding relationship between audio and video may bring, resulting in a limited identification range. Even if some methods use multi-modal information, they are limited to the binary classification of tampering or not, and cannot determine the specific tampered segments; for methods that use multi-modality to locate Deepfake time, there is no cross-reconstruction method to focus on the specific tampered area, and the identification and positioning accuracy is not high.

[0089] The embodiments of the present application provide a method, device, electronic device and storage medium for video authentication and time positioning. The technical solution of the present application includes: extracting features of the video and audio in the video file respectively to obtain input video features and input audio features aligned in the time dimension; cross-modal cross-reconstructing the input video features and input audio features, and using the spatial difference between true and false features to supervise the cross-modal cross-reconstruction step, thereby obtaining spatial error features; determining the probability of the corresponding current frame being tampered with according to the input video features, input audio features and spatial error features, and then classifying each current frame respectively; temporally positioning the tampered data frame in the video file according to the input video features, input audio features, spatial error features and the probability of the corresponding current frame being tampered with; wherein the data frame includes at least one of the current frame corresponding to the input video feature or the current frame corresponding to the input audio feature. The present application obtains the error between the audio and video modalities by cross-modal cross-reconstruction, thereby more accurately focusing on the specific tampered video frame, which can improve the accuracy of authentication and time positioning.

[0090] The embodiments of the present application provide a method, device, electronic device and storage medium for video authentication and time positioning, and relate to the field of artificial intelligence technology. The method, device, electronic device and storage medium for video authentication and time positioning provided by the embodiments of the present application can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured to provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and cloud servers for basic cloud computing services such as big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements a knowledge extraction method, etc., but is not limited to the above forms.

[0091] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0092] Reference Figure 1 The embodiment of the present application provides a video authentication and time positioning method, which may include but is not limited to S110 to S140, as follows:

[0093] S110: Extract features of the video and audio in the video file respectively to obtain input video features and input audio features aligned in time dimension.

[0094] In some examples, the present application may pre-process the video file before executing S110. Therefore, the embodiment of the present application may further include a step of pre-processing the video file in S100 before S110. Figure 2 , S100 may include the following steps S101 to S104:

[0095] S101: extracting the video and the audio from the video file;

[0096] S102: Fill the video and the audio respectively;

[0097] S103: scaling the size of the padded video and performing channel adjustment to obtain a video tensor of a first shape;

[0098] S104: extracting a mel-spectrogram from the padded audio, and further representing the mel-spectrogram as an audio tensor of a second shape.

[0099] Furthermore, S110 may further include:

[0100] Feature extraction is performed on the video tensor and the audio tensor respectively to obtain input video features and input audio features aligned in time dimension.

[0101] S120: Perform cross-modal cross-reconstruction on the input video features and the input audio features, and use the spatial difference between the true and false features to supervise the cross-modal cross-reconstruction step, so as to obtain spatial error features.

[0102] In some embodiments, the present application may further include a step of S200 training a cross-modal cross-reconstruction module, referring to Figure 3 The step of training the cross-modal cross-reconstruction module includes the following steps S201-S202:

[0103] S201: Train the V2A network using video features labeled with real and tampered video;

[0104] S202: training an A2V network using audio features annotated with real and tampered audio features; wherein the cross-modal cross-reconstruction module includes the V2A network and the A2V network; the A2V network and the V2A network are both implemented using a 4-layer standard transformer;

[0105] The expression of the loss function of the cross-modal cross-reconstruction module includes:

[0106]

[0107] P a =V2A(IN v );

[0108] P v =A2V(IN a );

[0109] Where N represents the number of video frames, P vi Indicates the features of the i-th video frame predicted using the input audio features, IN vi Represents the input video features; P ai Indicates the features of the i-th audio frame predicted using the input video features, IN ai Represents the audio features of the input;

[0110] The expression of the reconstruction supervision function of the cross-modal cross-reconstruction module includes:

[0111]

[0112] Mask pred_a =P a -Real a ;

[0113]

[0114] Mask ref_a =Fakea -Real a =IN a -Real a ;

[0115] Mask ref_v =Fake v -Real v =IN v -Real v ;

[0116] P a =V2A(IN v );

[0117] P v =A2V(IN a );

[0118] Among them, Mask pred_v Represents the difference between the input tampered audio features to reconstruct the predicted video features and the reference real video features; Masl ref Represents the difference between the input tampered video feature and the reference real video feature; Masl pred_a Mask represents the difference between the predicted audio features and the reference real audio features, which are reconstructed by inputting the tampered video features; ref_a represents the difference between the input tampered audio feature and the reference real audio feature; N represents the number of video frames, P v Fake represents the video features predicted using the input audio features. v =IN v Represents the input tampered video feature; P a Indicates the audio features predicted using the input video features, fake a =IN a Represents the tampered audio feature of the input; Real a Indicates the corresponding real audio features, Real v Indicates the corresponding real video features.

[0119] Reference Figure 4 , S120 may further include the following steps S121 to S123:

[0120] S121: Input the input video features to the V2A network, and use the V2A network to infer and obtain reconstructed audio features; input the input audio features to the A2V network, and use the A2V network to infer and obtain reconstructed video features;

[0121] S122: Subtracting the input video feature from the reconstructed video feature to obtain a reconstructed video error feature; subtracting the input audio feature from the reconstructed audio feature to obtain a reconstructed audio error feature;

[0122] S123: Input the reconstructed video error feature and the reconstructed audio error feature into a linear layer for linear weighting to obtain the spatial error feature.

[0123] S130: Determine the probability that the corresponding current frame has been tampered with according to the input video feature, the input audio feature and the spatial error feature, and then classify each of the current frames respectively.

[0124] Reference Figure 5 The embodiment of the present application may further include the following steps S131 to S133:

[0125] S131: When the training converges, the expected distribution of the reconstructed features is determined as follows:

[0126]

[0127] Among them, P a,i represents the features of the audio predicted using the input video features; v,i Fake represents the features of the video predicted using the input audio features; a,i Fake represents the features of the corresponding tampered audio frames in the dataset used for training. v,i Represents the features of the corresponding tampered video frames in the dataset used for training;

[0128] S132: After the training converges, the distribution of the spatial error features output after the difference is determined as follows:

[0129]

[0130] Among them, Out a,i The audio feature output by the cross-modal cross-reconstruction module is used as the first spatial error; Out v,i The video features output by the cross-modal cross-reconstruction module are used as the second spatial error; the first spatial error and the second spatial error are used for subsequent audio and video frame classification and time positioning tasks;

[0131] It should be noted that the output is close to the 0 vector when and only when the audio and video are both true, otherwise it follows a specific distribution; only when there is no tampering in the audio and video modalities, the cross-reconstruction module outputs the feature Out a and Out vtends to 0, and as long as any mode is tampered, both are not 0, thereby distinguishing the real audio and video input features from the tampered audio and video input features;

[0132] The step of classifying each current frame separately comprises the following steps:

[0133] S133: performing frame classification on each of the current frames according to the frame classification loss function;

[0134] The loss function of the frame classification includes:

[0135]

[0136] Among them, Lossf a 、Lossf v 、Lossf s Respectively represent the loss functions for frame classification using the input audio feature, the input video feature and the spatial error feature; is the standard cross entropy loss function.

[0137] S140: Temporally locate the tampered data frame in the video file according to the input video feature, the input audio feature, the spatial error feature and the probability that the corresponding current frame has been tampered with; wherein the data frame includes at least one of the current frame corresponding to the input video feature or the current frame corresponding to the input audio feature.

[0138] Further, S140 may include the following steps:

[0139] Using a time positioning loss function, temporally positioning the tampered data frame in the video file according to the input video feature, the input audio feature, the spatial error feature, and the probability that the current frame has been tampered with;

[0140] The loss function of the time positioning is:

[0141]

[0142] Among them, Lossb a As the boundary matching loss functions for audio tampering time positioning, video tampering time positioning and any modality tampering time positioning respectively; N is the number of video frames; D is the maximum number of frames of the tampered data frame segment; represents the confidence of the tampered data frame segment starting from the i-th frame and ending at the i+j-th frame, Y b a,i j represents the corresponding true value.

[0143] As a further implementation, the embodiment of the present application may further include the following steps:

[0144] S150: Visually display the tampered data frame and the corresponding time location.

[0145] Exemplarily, at least one of the audio frames or video frames determined to be tampered with and the corresponding tampering time are visualized, for example, by means of text, image, video or voice.

[0146] Next, the solution of the embodiment of the present application will be introduced and explained in detail with reference to specific application examples.

[0147] The purpose of this embodiment is to provide a multimodal Deepfake video authentication and temporal positioning method based on audio and video cross-reconstruction. Through the spatial error of audio and video cross-reconstruction, the authentication model can focus on the specific tampering area. The specific cross-reconstruction module uses the cross-modal guided contrast loss function provided by this embodiment in the training stage to distinguish the real number from the tampered data. Specifically, this embodiment includes the following key technical features:

[0148] (1) By using cross-modal cross-reconstruction, the model focuses on the specific tampered area, generating cross-modal spatial errors for frame-level classification and temporal positioning, thereby improving the identification and temporal positioning capabilities of the anti-counterfeiting model.

[0149] (2) A novel cross-modal guided contrast loss function is proposed. When using the input features for cross-modal reconstruction, the reference information of the other modality is used to guide the reconstruction process and distinguish the true and false audio and video features at the feature level.

[0150] Reference Figure 6 , this embodiment provides an example flow chart of a video authentication and time positioning method.

[0151] Specifically, this embodiment may include the following steps:

[0152] Step 1. Preprocessing of video files:

[0153] For the video files to be detected (including video and audio), they are first preprocessed to unify the lengths of audio and video files of different lengths. The steps are as follows:

[0154] 1.1 Padding: First, pad the input audio and video. Pad the video image to frame_padding = 512 frames, and pad the audio file to audio_padding = int (frame_padding / fps*sps) sample points, where the video frame rate fps = 25, the audio sampling rate sps = 16000, and the total number of audio samples audio_padding is filled to 327680.

[0155] 1.2 Dimension adjustment: For the padded video, resize it to the size of (96, 96) and adjust the channels. The final video input is represented as a tensor of shape (3, 512, 96, 96), where 3 is the number of RGB channels, 512 is the number of frames, and 96 is the height and width of the video.

[0156] For the padded audio, use torchaudio.transforms.MelSpectrogram to extract the Mel spectrogram, the window size of the spectrogram n_fft = 321, the step size step = 160. Finally, an audio input is represented as a tensor of shape (64, 2048), where 64 is the number of Mel filters and 2048 is the "number of frames" (width) of the spectrogram.

[0157] Step 2. Feature extraction and alignment:

[0158] For the preprocessed video tensor, use MViTv2-Base as the video encoder to extract features and obtain a feature vector with a shape of (256, 512), where 256 is the video vector dimension and 512 is the number of frames;

[0159] For the preprocessed audio tensor, ViT-Base is used as the audio encoder for feature extraction to obtain a feature vector with a shape of (256, 512), where 256 is the audio vector dimension and 512 is the number of frames; at this point, the video and audio features are aligned in the time feature dimension.

[0160] Step 3. Cross-modality reconstruction:

[0161] For the audio and video features that have passed through the video encoder and audio encoder, the A2V network and V2A network are used for cross-prediction reconstruction. Specifically, both A2V and V2A are implemented using a 4-layer standard transformer to implement the attention mechanism in the time dimension, giving different attention to the real part and the tampered part in a video.

[0162] In the training phase, two types of data in the LAV-DF dataset are used for training, such as Figure 7As shown:

[0163] (1) Completely authentic video, with no tampering in audio or video modalities

[0164] (2) Completely tampered video, the audio and video are tampered synchronously, and the specific tampered time segment and the original video are provided. This embodiment uses the timestamp provided by LAV-DF to locate the tampered segment corresponding to the corresponding segment in the real video.

[0165] During training, this embodiment uses different loss functions for real audio and video frames and tampered audio and video frames:

[0166] For real audio and video frames, this embodiment uses its own cross-modal real data to perform cross-reconstruction supervision. That is, use real video to predict real audio, and use real audio to predict real video. The loss function is as follows:

[0167]

[0168] P a =V2A(IN v );

[0169] P v =A2V(IN a );

[0170] Where N represents the number of video frames, P vi represents the features of the i-th video frame predicted using the input audio, IN vi Represents the input video feature frame; P ai represents the features of the i-th audio frame predicted using the input video, IN ai Features representing the input real audio frame.

[0171] For audio and video that are tampered with at the same time, this embodiment uses its own cross-modal false data and the corresponding real data for cross-reconstruction supervision. Specifically, for a known false audio feature input, this embodiment uses it to predict the video feature, and then subtracts it from a reference real video frame. The difference is called Mask. pred_v , another Mask used for supervision uses the input fake video to make a difference with a reference real video feature, and the difference is called Mask ref , by minimizing Mask pred_V With 2x Mask ref_v The difference between them realizes the supervision of the spatial error of the visual modality.

[0172] Similarly, for a known false video feature input, this embodiment uses it to predict the audio feature, and then subtracts it from a reference real audio feature. The difference is called Mask. pred_a , another Mask for supervision uses the input false audio features to make a difference with a reference real audio feature, and the difference is called Mask ref_a , by minimizing Mask prrd_a With 2x Mask ref_a The difference between them realizes the supervision of the spatial error of the auditory modality; the relevant loss function is as follows:

[0173]

[0174] Mask pred_a =P a -Real a ;

[0175]

[0176] Mask ref_a =Fake a -Real a =IN a -Real a ;

[0177] Mask ref_v =Fake v -Real v =IN v -Real v ;

[0178] P a =V2A(IN v );

[0179] P v =A2V(IN a );

[0180] Where N represents the number of video frames, P v Represents the features of the video predicted using the input audio, Fake v =IN v Represents the input tampered video feature; P a Fake represents the features of the audio predicted using the input video. a =IN a Represents the tampered audio feature of the input, Real a Indicates the corresponding real audio features, Real v Indicates the corresponding real video features.

[0181] In the inference stage, the input video feature is used to predict an audio feature, which is then subtracted from the input audio feature to serve as the output reconstructed audio error feature; similarly, the input audio feature is used to predict a video feature, which is then subtracted from the input video feature to serve as the output reconstructed video error feature. The relevant expressions are as follows:

[0182] Out a =P a -IN a ;

[0183] Out v =P v -IN v ;

[0184] Out combined =Linear(Out a ,Out v );

[0185] Among them, Linear represents a linear layer that accepts Out a and Out v As input, the output dimension is linearly weighted and Out a and Out v The same feature vector. The inference phase is as follows Figure 8 shown.

[0186] Theoretical analysis: When training converges, the real audio input features tend to predict the real video input features, and the real video input features tend to predict the real audio input features; the tampered audio input tends to predict the tampered video input plus Mask ref_v , the tampered video input tends to predict the tampered audio input plus Mask ref_a ;Right now:

[0187]

[0188]

[0189] Among them, Fake a,i Represents the features of the corresponding tampered audio frames in the dataset, Fake v,i Represents the features of the corresponding tampered video frame in the data.

[0190] The following expression can be derived by considering the true or false of the audio and video input:

[0191]

[0192] From the above formula, we can see that only when there is no tampering in the audio and video modes, the cross-reconstruction module outputs the feature Outa and Out v tends to 0, and as long as any mode is tampered with, both of them are not 0, thereby distinguishing the real audio and video input features from the tampered audio and video input features.

[0193] Step 4. Frame classification:

[0194] The frame classification module is implemented using a standard one-dimensional convolution, which regresses the probability that the frame is tampered with based on the features of the current frame.

[0195] Still refer to Figure 6 In this embodiment, three independent frame-level classifiers are used to classify pure audio tampering, pure visual tampering, and audio or video tampering. The specific loss function is as follows:

[0196]

[0197] Among them, Lossf a Lossf v Lossf s Respectively represent the loss functions for frame classification using audio features, visual features, and spatial error features, is the standard cross entropy loss function.

[0198] Step 5. Time positioning:

[0199] Temporal localization consists of three relatively independent temporal localization modules, which respectively use the audio features output by the audio encoder, the video features output by the video encoder, the spatial error features output by the cross-modal cross-reconstruction model, and the corresponding frame classification results to perform the temporal localization task. Specifically, the temporal localization module mainly includes the boundary matching candidate generation module and the temporal attention transformer module. The boundary matching candidate generation module generates a tensor of shape [T, N, D, T]. The relevant code is as follows:

[0200] #(B,n_feature_in,temporal_dim)->(B,n_feature_in,sample,max_duration,temporal_dim);

[0201] The loss function is as follows:

[0202]

[0203] Among them, Lossb a It can be used as the boundary matching loss function for the temporal positioning of audio tampering, visual tampering and any modal tampering, respectively. N is the number of frames in the video, and D is the maximum possible length of the tampered segment (frames). Y represents the confidence level of the tampered segment starting from the i-th frame and ending at the i+j-th frame. b a,j j represents the corresponding true value.

[0204] The beneficial effects of this embodiment include:

[0205] By focusing on the cross-reconstruction error, the model can focus on the specific tampering area, thereby improving the accuracy of authentication and time positioning.

[0206] A new supervised loss function for cross-modal cross-reconstruction. For cross-modal cross-reconstruction, when it is known that the current modality has been tampered with, the real features and tampered features of the other modality are used as supervisory signals to distinguish the real signal from the tampered signal during reconstruction.

[0207] In summary, this embodiment provides a multimodal Deepfake video authentication and time positioning method based on audio and video cross-reconstruction, which uses cross-modal information to improve the identification and time positioning capabilities of audio and video tampering, and provides more powerful tools for the development and application of network security and forensic identification to cope with the ever-evolving Deepfake technology.

[0208] Reference Fig. 9 The embodiment of the present application further provides a video authentication and time positioning device, which can implement the above-mentioned video authentication and time positioning method, and the device includes:

[0209] A feature extraction unit, used to extract features of the video and audio in the video file respectively, to obtain input video features and input audio features aligned in time dimension;

[0210] A feature reconstruction unit, configured to perform cross-modal cross-reconstruction on the input video feature and the input audio feature, and supervise the cross-modal cross-reconstruction step by using the spatial difference between the true and false features, thereby obtaining a spatial error feature;

[0211] A feature classification unit, used to determine the probability that the corresponding current frame is tampered with according to the input video feature, the input audio feature and the spatial error feature, and then classify each of the current frames respectively;

[0212] A time positioning unit is used to temporally locate the tampered data frame in the video file based on the input video feature, the input audio feature, the spatial error feature and the probability that the corresponding current frame has been tampered with; wherein the data frame includes at least one of the current frame corresponding to the input video feature or the current frame corresponding to the input audio feature.

[0213] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0214] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned video authentication and time positioning method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0215] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0216] See also Fig.10 , Fig.10 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0217] The processor 1001 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0218] The memory 1002 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1002 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1002, and the processor 1001 calls and executes the video authentication and time positioning method of the embodiment of this application;

[0219] Input / output interface 1003, used to implement information input and output;

[0220] The communication interface 1004 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0221] A bus 1005 , which transmits information between various components of the device (e.g., the processor 1001 , the memory 1002 , the input / output interface 1003 , and the communication interface 1004 );

[0222] The processor 1001 , the memory 1002 , the input / output interface 1003 and the communication interface 1004 are connected to each other in communication within the device via the bus 1005 .

[0223] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned video authentication and time positioning method is implemented.

[0224] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0225] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0226] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0227] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0228] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0229] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0230] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0231] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0232] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0233] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0234] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0235] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0236] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A video authentication and time positioning method, characterized in that: The method comprises the following steps: Extract features of the video and audio in the video file respectively to obtain input video features and input audio features aligned in the time dimension; Performing cross-modal cross-reconstruction on the input video features and the input audio features, and using the spatial difference between the true and false features to supervise the cross-modal cross-reconstruction step, thereby obtaining a spatial error feature; Determine the probability that the corresponding current frame is tampered with according to the input video feature, the input audio feature and the spatial error feature, and then classify each of the current frames respectively; The tampered data frame in the video file is temporally located based on the input video feature, the input audio feature, the spatial error feature and the probability that the corresponding current frame has been tampered with; wherein the data frame includes at least one of the current frame corresponding to the input video feature or the current frame corresponding to the input audio feature.

2. The video authentication and time positioning method according to claim 1, characterized in that: Before extracting features from the video and audio in the video file to obtain input video features and input audio features aligned in the time dimension, the method further includes a step of preprocessing the video file, and the step of preprocessing the video file includes the following steps: Extract the video and the audio from the video file; Filling the video and the audio respectively; Scaling the size of the padded video and performing channel adjustment to obtain a video tensor of a first shape; Extracting a Mel-spectrogram from the padded audio, and then representing the Mel-spectrogram as an audio tensor of a second shape; The step of extracting features of the video and audio in the video file respectively to obtain input video features and input audio features aligned in the time dimension comprises the following steps: Feature extraction is performed on the video tensor and the audio tensor respectively to obtain input video features and input audio features aligned in time dimension.

3. The video authentication and time positioning method according to claim 1, characterized in that: Before the step of cross-modal reconstruction of the input video features and the input audio features and supervising the cross-modal reconstruction using the spatial difference between the true and false features to obtain the spatial error features, the method further includes the step of training a cross-modal cross-reconstruction module, and the step of training the cross-modal cross-reconstruction module includes the following steps: The V2A network is trained using video features labeled with real and tampered video features; the A2V network is trained using audio features labeled with real and tampered audio features; wherein the cross-modal cross-reconstruction module includes the V2A network and the A2V network; the A2V network and the V2A network are both implemented using a 4-layer standard transformer; The expression of the loss function of the cross-modal cross-reconstruction module includes: P a =V2A(IN v ); P v =A2V(IN a ); Where N represents the number of video frames, P vi Indicates the features of the i-th video frame predicted using the input audio features, IN vi Represents the input video features; P ai Indicates the features of the i-th audio frame predicted using the input video features, IN ai Represents the audio features of the input; The expression of the reconstruction supervision function of the cross-modal cross-reconstruction module includes: Mask pred_a =P a -Real a ; Mask ref_a =Fake a -Real a =IN a -Real a ; Mask ref_v =Fake v -Real v =IN v -Real v ; P a =V2A(IN v ); P v =A2V(IN a ); Among them, Mask pred_v Mask represents the difference between the predicted video features and the reference real video features, which are reconstructed by inputting the tampered audio features to obtain the predicted video features. ref Mask represents the difference between the input tampered video features and the reference real video features; pred_a Mask represents the difference between the predicted audio features and the reference real audio features, which are reconstructed by inputting the tampered video features; ref_a represents the difference between the input tampered audio feature and the reference real audio feature; N represents the number of video frames, P v Fake represents the video features predicted using the input audio features. v =IN v Represents the input tampered video feature; P a Indicates the audio features predicted using the input video features, Fake a =IN a Represents the tampered audio feature of the input; Real a Indicates the corresponding real audio features, Real b Indicates the corresponding real video features.

4. The video authentication and time positioning method according to claim 3, characterized in that: The step of performing cross-modal cross reconstruction on the input video features and the input audio features and supervising the cross-modal cross reconstruction using the spatial difference between the true and false features to obtain the spatial error features comprises the following steps: Input the input video features into the V2A network, and use the V2A network to infer and obtain reconstructed audio features; input the input audio features into the A2V network, and use the A2V network to infer and obtain reconstructed video features; Subtracting the input video feature from the reconstructed video feature to obtain a reconstructed video error feature; subtracting the input audio feature from the reconstructed audio feature to obtain a reconstructed audio error feature; The reconstructed video error feature and the reconstructed audio error feature are input into a linear layer for linear weighting to obtain the spatial error feature.

5. The video authentication and time positioning method according to claim 3, characterized in that: The method further comprises the following steps: When the training converges, the expected distribution of the reconstructed features is determined as follows: Among them, P a，i represents the features of the audio predicted using the input video features; v，i Fake a，i Fake represents the features of the corresponding tampered audio frames in the training dataset. v，i Represents the features of the corresponding tampered video frames in the dataset used for training; Then, when the training converges, the distribution of the spatial error characteristics output after the difference is determined as follows: Among them, Out a，i The audio feature output by the cross-modal cross-reconstruction module is used as the first spatial error; Out v，i The video features output by the cross-modal cross-reconstruction module are used as the second spatial error; the first spatial error and the second spatial error are used for subsequent audio and video frame classification and time positioning tasks; The step of classifying each current frame separately comprises the following steps: Performing frame classification on each of the current frames according to the loss function of frame classification; The loss function of the frame classification includes: Among them, Lossf a 、Lossf v 、Lossf s Respectively represent loss functions for frame classification using the input audio feature, the input video feature, and the spatial error feature; is the standard cross entropy loss function.

6. The video authentication and time positioning method according to claim 1, characterized in that: The step of temporally locating the tampered data frame in the video file according to the input video feature, the input audio feature, the spatial error feature, and the probability that the current frame has been tampered with comprises the following steps: Performing time positioning of the tampered data frame in the video file according to the input video feature, the input audio feature, the spatial error feature, and the probability that the current frame has been tampered with using a time positioning loss function; The loss function of the time positioning is: Among them, Lossb a As the boundary matching loss functions for audio tampering time positioning, video tampering time positioning and any modality tampering time positioning respectively; N is the number of video frames; D is the maximum number of frames of the tampered data frame segment; represents the confidence of the tampered data frame segment starting from the i-th frame and ending at the i+j-th frame, Y b a，ij Represents the corresponding true value.

7. The video authentication and time positioning method according to any one of claims 1 to 6, characterized in that: The method further comprises the following steps: The tampered data frame and the corresponding time location are visually displayed.

8. Video authentication and time positioning device, characterized in that: The device comprises: A feature extraction unit, used to extract features of the video and audio in the video file respectively, to obtain input video features and input audio features aligned in time dimension; A feature reconstruction unit, configured to perform cross-modal cross-reconstruction on the input video feature and the input audio feature, and supervise the cross-modal cross-reconstruction step by using the spatial difference between the true and false features, thereby obtaining a spatial error feature; A feature classification unit, used to determine the probability that the corresponding current frame is tampered with according to the input video feature, the input audio feature and the spatial error feature, and then classify each of the current frames respectively; A time positioning unit is used to temporally locate the tampered data frame in the video file based on the input video feature, the input audio feature, the spatial error feature and the probability that the corresponding current frame has been tampered with; wherein the data frame includes at least one of the current frame corresponding to the input video feature or the current frame corresponding to the input audio feature.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the video authentication and time positioning method as claimed in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the video authentication and time positioning method according to any one of claims 1 to 7 is implemented.

Images