Video authentication method and device based on image processing, and electronic equipment

Abstract

The invention discloses a video authentication method and device based on image processing and electronic equipment, and relates to the technical field of image processing, and the method comprises the steps: obtaining a coded video to be authenticated, and dividing the coded video to be authenticated into N overlapped video frame groups; calculating the residual noise of each P frame in the overlapped video frame group, and comparing the residual noise of each P frame in the overlapped video frame group with the error value corresponding to the P frame to obtain a first comparison result of the overlapped video frame group; determining that the overlapped video frame group does not have intra-frame counterfeiting based on the first comparison result; under the condition that the intra-frame counterfeiting does not exist in the video to be authenticated, calculating a similarity value of every two overlapped video frame groups in the video to be authenticated; and comparing the to-be-authenticated video with a similarity threshold to obtain a second comparison result, and determining an authentication result of the to-be-authenticated video based on the second comparison result. According to the invention, a technical problem that the detection efficiency is low because video frames in a video need to be detected frame by frame when authentication processing is carried out on the video in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a video authentication method and device based on image processing, and an electronic device. Background Art

[0002] With the rapid iteration of network technology, network security has become an increasingly important topic. Video authentication has become an important means of identity authentication in digital service platforms due to its intuitiveness and difficulty in forging. For example, in the field of education, with the rise of smart campuses, video authentication has been widely used in the login process of campus platforms. In the financial field, video authentication has gradually developed into one of the most important means of user identity authentication to ensure the authenticity and security of user identity and protect user property safety.

[0003] However, with the development of artificial intelligence technology, especially the application of deep learning in image processing, video tampering technologies such as face swapping and video synthesis have become increasingly mature, which has brought unprecedented challenges to traditional video authentication methods. Illegal users can use these technologies to forge authentication videos to make them look very similar to real users, thereby bypassing security verification. In addition, traditional video authentication often relies on frame-by-frame face recognition or action recognition. This method not only consumes a lot of computing resources and has low detection efficiency, but also significantly reduces the system's processing speed and response time when faced with a large number of concurrent login requests, affecting the user experience.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a video authentication method and device based on image processing, and an electronic device, so as to at least solve the technical problem in the related art that when authenticating a video, it is necessary to detect each video frame in the video frame by frame, resulting in low detection efficiency.

[0006] According to one aspect of an embodiment of the present invention, a video authentication method based on image processing is provided, comprising: obtaining an encoded video to be authenticated, and dividing the video to be authenticated into N overlapping video frame groups, wherein the video to be authenticated is encoded based on a secondary compression algorithm, and the overlapping video frame group indicates that two adjacent video frame groups contain M repeated video frames, and M and N are both positive integers; for each overlapping video frame group, calculating the residual noise of each P frame in the overlapping video frame group, and comparing the residual noise of each P frame in the overlapping video frame group with the error value corresponding to the P frame, to obtain a first comparison result of the overlapping video frame group, wherein the P frame represents a forward prediction Measure frames, the forward prediction frame represents a video frame containing pixel differences between the current frame and the previous frame; when the first comparison result indicates that the residual noise of all P frames in the overlapping video frame group is less than or equal to the error value corresponding to the P frame, determine that there is no intra-frame forgery in the overlapping video frame group; when there is no intra-frame forgery in N overlapping video frame groups in the video to be authenticated, calculate the similarity value of each two overlapping video frame groups in the video to be authenticated; compare the similarity value of each two overlapping video frame groups in the video to be authenticated with a similarity threshold to obtain a second comparison result, and determine the authentication result of the video to be authenticated based on the second comparison result.

[0007] Optionally, for each overlapping video frame group, the step of calculating the residual noise of each P frame in the overlapping video frame group includes: for each overlapping video frame group, obtaining an I frame in the overlapping video frame group, wherein the I frame is a key frame in the video to be authenticated, and the key frame is a video frame containing complete image data; denoising the I frame based on wavelet transform to obtain a denoised I frame; calculating the difference data between the denoised I frame and the I frame before denoising to obtain the residual noise of the denoised I frame; and calculating the residual noise of the P frame adjacent to the I frame based on the residual noise of the denoised I frame.

[0008] Optionally, the step of denoising the I frame based on wavelet transform to obtain the denoised I frame includes: decomposing the I frame to obtain grayscale images of the I frame in red RGB channel, green RGB channel and blue RGB channel respectively; performing wavelet transform on the grayscale image of the red RGB channel, the grayscale image of the green RGB channel and the grayscale image of the blue RGB channel of the I frame respectively to obtain a frequency subband set corresponding to the I frame, wherein the frequency subband set includes K frequency subbands of the I frame, and the frequency subband is the frequency component of the I frame in the target direction under the target color channel; for each of the frequency subbands, calculating the local variance of the frequency subband; denoising the frequency subband based on the local variance to obtain the denoised frequency subband, and calculating the wavelet coefficients of the denoised frequency subband; performing inverse wavelet transform on the denoised wavelet coefficients of all the frequency subbands to obtain the denoised I frame.

[0009] Optionally, the step of calculating the residual noise of the P frame adjacent to the I frame based on the residual noise of the denoised I frame includes: for any P frame, selecting the I frame adjacent to the P frame as a reference frame; for each area block in the P frame, obtaining a target area block matching the area block in the P frame in the reference frame; obtaining the pixel difference between the area block in the P frame and the target area block in the reference frame through motion compensation; calculating the residual noise of each area block in the P frame based on the pixel difference between the area block and the target area block and the residual noise of the denoised reference frame on each target area block, so as to obtain the residual noise of the P frame.

[0010] Optionally, the step of calculating the similarity value of each two overlapping video frame groups in the video to be authenticated includes: for each overlapping video frame, combining two adjacent video frames in the overlapping video frames to obtain a video frame pair; calculating the noise correlation value of the video frame pair, and splicing the noise correlation values ​​of all video frame pairs in the overlapping video frames to obtain a noise vector of the overlapping video frame; calculating the similarity value based on the noise vectors of any two overlapping video frames in the video to be authenticated, to obtain the similarity value of each two overlapping video frame groups in the video to be authenticated.

[0011] Optionally, after obtaining the first comparison result of the overlapping video frame group, it also includes: when the first comparison result indicates that the residual noise of any P frame in the overlapping video frame group is greater than the error value, determining that there is intra-video frame forgery in the overlapping video frame; determining that there is an abnormality in the video to be authenticated based on the overlapping video frame with intra-video frame forgery, generating an authentication result of authentication failure for the video to be authenticated with the abnormality, and returning the authentication result to the user end.

[0012] Optionally, the step of determining the authentication result of the video to be authenticated based on the second comparison result includes: when the second comparison result indicates that the similarity value of any two overlapping video frame groups in the video to be authenticated is greater than a similarity threshold, determining that there is video frame forgery between the two overlapping video frame groups; determining that there is an abnormality in the video to be authenticated based on the overlapping video frame groups with video frame forgery between them, generating an authentication result of authentication failure for the video to be authenticated with the abnormality, and returning the authentication result to the user end.

[0013] Optionally, the calculation formula of the error value is expressed as: in, A represents the error value corresponding to the P frame, represents the error value percentage corresponding to the P frame, S represents the change area block between the P frame and the adjacent video frame, K represents the Kth change area block, K represents the average residual noise in the Kth change area block, and I represents the average residual noise of the P frame.

[0014] According to another aspect of an embodiment of the present invention, a video authentication device based on image processing is also provided, comprising: an acquisition unit, used to acquire an encoded video to be authenticated, and divide the video to be authenticated into N overlapping video frame groups, wherein the video to be authenticated is encoded based on a secondary compression algorithm, and the overlapping video frame group represents M repeated video frames between two adjacent video frame groups, and M and N are both positive integers; a first calculation unit, used to calculate the residual noise of each P frame in the overlapping video frame group for each overlapping video frame group, and compare the residual noise of each P frame in the overlapping video frame group with the error value corresponding to the P frame to obtain a first comparison result of the overlapping video frame group, wherein the P frame represents a forward prediction frame, The forward prediction frame represents a video frame containing pixel differences between the current frame and the previous frame; a determination unit is used to determine that there is no intra-frame forgery in the overlapping video frame group when the first comparison result indicates that the residual noise of all P frames in the overlapping video frame group is less than or equal to the error value corresponding to the P frame; a second calculation unit is used to calculate the similarity value of each two overlapping video frame groups in the video to be authenticated when there is no intra-frame forgery in N overlapping video frame groups in the video to be authenticated; a comparison unit is used to compare the similarity value of each two overlapping video frame groups in the video to be authenticated with a similarity threshold to obtain a second comparison result, and determine the authentication result of the video to be authenticated based on the second comparison result.

[0015] Optionally, the first calculation unit includes: a first acquisition module, used to obtain, for each overlapping video frame group, an I frame in the overlapping video frame group, wherein the I frame is a key frame in the video to be authenticated, and the key frame is a video frame containing complete image data; a first denoising module, used to denoise the I frame based on wavelet transform to obtain a denoised I frame; a first calculation module, used to calculate the difference data between the denoised I frame and the I frame before denoising to obtain the residual noise of the denoised I frame; and a second calculation module, used to calculate the residual noise of the P frame adjacent to the I frame based on the residual noise of the denoised I frame.

[0016] Optionally, the first denoising module also includes: a first decomposition submodule, used to decompose the I frame to obtain grayscale images of the I frame in the red RGB channel, the green RGB channel and the blue RGB channel respectively; a first transformation submodule, used to perform wavelet transform on the grayscale image of the red RGB channel, the grayscale image of the green RGB channel and the grayscale image of the blue RGB channel of the I frame respectively to obtain a frequency subband set corresponding to the I frame, wherein the frequency subband set includes K frequency subbands of the I frame, and the frequency subband is the frequency component of the I frame in the target direction under the target color channel; a first calculation submodule, used to calculate the local variance of the frequency subband for each of the frequency subbands; a first denoising submodule, used to perform denoising on the frequency subband based on the local variance to obtain the denoised frequency subband, and calculate the wavelet coefficients of the denoised frequency subband; a second transformation submodule, used to perform inverse wavelet transform on the denoised wavelet coefficients of all the frequency subbands to obtain the denoised I frame.

[0017] Optionally, the first calculation module includes: a first selection submodule, used for selecting an I frame adjacent to the P frame as a reference frame for any P frame; a first acquisition submodule, used for acquiring, for each area block in the P frame, a target area block that matches the area block in the P frame in the reference frame; a second acquisition submodule, used for acquiring the pixel difference between the area block in the P frame and the target area block in the reference frame through motion compensation; and a second calculation submodule, used for calculating the residual noise of each area block in the P frame based on the pixel difference between the area block and the target area block and the residual noise of the denoised reference frame on each target area block, so as to obtain the residual noise of the P frame.

[0018] Optionally, the second calculation unit includes: a first combination module, used to combine two adjacent video frames in the overlapping video frames for each overlapping video frame to obtain a video frame pair; a third calculation module, used to calculate the noise correlation value of the video frame pair, and splice the noise correlation values ​​of all video frame pairs in the overlapping video frames to obtain a noise vector of the overlapping video frame; and a fourth calculation module, used to calculate the similarity value based on the noise vectors of any two overlapping video frames in the video to be authenticated, and obtain the similarity value of each two overlapping video frame groups in the video to be authenticated.

[0019] Optionally, the image processing-based video authentication device also includes: a first determination module, used to determine that there is intra-video frame forgery in the overlapping video frame when the first comparison result indicates that the residual noise of any P frame in the overlapping video frame group is greater than the error value; a first generation module, used to determine that there is an abnormality in the video to be authenticated based on the overlapping video frame with intra-video frame forgery, generate an authentication result of authentication failure for the video to be authenticated with the abnormality, and return the authentication result to the user end.

[0020] Optionally, the determination unit includes: a second determination module, used to determine that there is video frame forgery between two overlapping video frame groups when the second comparison result indicates that the similarity value of any two overlapping video frame groups in the video to be authenticated is greater than a similarity threshold; a third determination module, used to determine that there is an abnormality in the video to be authenticated based on the overlapping video frame groups with video frame forgery, generate an authentication result of authentication failure for the video to be authenticated with the abnormality, and return the authentication result to the user end.

[0021] Optionally, the calculation formula of the error value is expressed as: in, A represents the error value corresponding to the P frame, A represents the error value percentage corresponding to the P frame, S represents the change area block between the P frame and the adjacent video frame, K represents the Kth change area block, K represents the average residual noise in the Kth change area block, and I represents the average residual noise of the P frame.

[0022] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above-mentioned video authentication methods based on image processing.

[0023] In the present application, the following steps are performed: first, the encoded video to be authenticated is obtained, and the video to be authenticated is divided into N overlapping video frame groups, wherein the video to be authenticated is encoded based on a secondary compression algorithm, and the overlapping video frame group indicates that two adjacent video frame groups contain M repeated video frames, and M and N are both positive integers, and then for each overlapping video frame group, the residual noise of each P frame in the overlapping video frame group is calculated, and the residual noise of each P frame in the overlapping video frame group is compared with the error value corresponding to the P frame, so as to obtain the first comparison result of the overlapping video frame group, wherein the P frame represents a forward prediction frame, and the forward prediction The measured frame represents a video frame containing pixel differences between the current frame and the previous frame, and when the first comparison result indicates that the residual noise of all P frames in the overlapping video frame group is less than or equal to the error value corresponding to the P frame, it is determined that there is no intra-frame forgery in the overlapping video frame group, and when there is no intra-frame forgery in N overlapping video frame groups in the video to be authenticated, the similarity value of each two overlapping video frame groups in the video to be authenticated is calculated, and finally the similarity value of each two overlapping video frame groups in the video to be authenticated is compared with the similarity threshold to obtain a second comparison result, and the authentication result of the video to be authenticated is determined based on the second comparison result.

[0024] In this application, when authenticating a video, the video to be authenticated is divided into N overlapping video frame groups, and the overlapping video frame groups are used as detection objects. The residual noise of the P frames in the group is calculated and compared with the preset error value, which can accurately detect the tampering within the frame. At the same time, the similarity value between the video frame groups is calculated and compared with the set similarity threshold, which can detect the tampering between frames. There is no need to detect the video frame by frame, which can effectively reduce the amount of calculation, improve the detection efficiency, and maintain high-precision forgery detection within and between frames. This solves the technical problem in the related art that when authenticating a video, the video frames in the video need to be detected frame by frame, and the detection efficiency is low. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0026] Figure 1 is a flow chart of an optional video authentication method based on image processing according to an embodiment of the present invention;

[0027] Figure 2 is a schematic diagram of an optional overlapping video frame group according to an embodiment of the present invention;

[0028] Figure 3 is a schematic diagram of an optional video authentication device based on image processing according to an embodiment of the present invention;

[0029] Figure 4 The present invention is a hardware structure block diagram of an electronic device (or mobile device) for a video authentication method based on image processing according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] It should be noted that the image processing-based video authentication method and device in the present application can be used in the field of image processing technology when security authentication of videos is performed based on image processing, and can also be used in any field outside the field of image processing technology when security authentication of videos is performed based on image processing. The application does not limit the application field of the image processing-based video authentication method and device.

[0033] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0034] It should be noted that in this application, when collecting and analyzing customer information, corresponding operation entrances are provided for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered.

[0035] The following embodiments of the present invention can be applied to various video authentication systems / applications / devices based on image processing. The present invention divides the video to be authenticated into multiple overlapping video frame groups, takes the overlapping video frame groups as the research object, calculates the residual noise of the P frames after secondary compression in the group, and detects forgery in the video frame by comparing the residual noise and the error value. At the same time, similarity calculation is performed on two overlapping video frame groups, and forgery between video frames is detected by comparing the similarity value and the similarity threshold. The accuracy of authentication and recognition can be guaranteed while efficient detection and authentication are performed.

[0036] The present invention is described in detail below in conjunction with various embodiments.

[0037] Embodiment 1

[0038] According to an embodiment of the present invention, an embodiment of a video authentication method based on image processing is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0039] Figure 1 is a flowchart of an optional video authentication method based on image processing according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0040] Step S101, obtaining the encoded video to be authenticated, and dividing the video to be authenticated into N overlapping video frame groups.

[0041] It should be noted that with the rapid iteration of network technology, network security has received more and more attention, and current security authentication methods are emerging in an endless stream. Video authentication, as a method of authentication that is more difficult to forge, has gradually become the mainstream of current security authentication, especially for some fields with higher security requirements, such as campus security platforms, financial service platforms, etc. However, current video authentication also has the risk of forgery. For example, facial authentication videos are forged through AI face-changing technology or synthetic video frames, which leads to hidden dangers in network security.

[0042] It should be noted that video forgery includes intra-frame forgery and inter-frame forgery. Intra-frame forgery includes tampering with, deleting, and adding specific content of video frames. Inter-frame forgery is to modify the position of video frames by copying and pasting video frames, or to synthesize a new video from different video frames. When authenticating a video, it is necessary to detect and authenticate both intra-frame forgery and inter-frame forgery.

[0043] It should be noted that in order to avoid network security risks, it is necessary to accurately authenticate the video to be authenticated to ensure the authenticity of the video to be authenticated. In order to avoid the problems of long detection time and low detection efficiency caused by frame-by-frame analysis, which affects the user experience, the embodiment of the present invention encodes the video to be authenticated based on a secondary compression algorithm, and divides the encoded video into multiple overlapping video frame groups. Intra-frame and inter-frame forgery authentication are performed on the overlapping video frame groups, which can ensure the security of the video to be authenticated while quickly authenticating.

[0044] It should be noted that the video to be authenticated in the embodiment of the present invention can be a video composed of a GOP image group obtained by secondary compression encoding. The user records the authentication video through a mobile terminal, and the authentication video is encoded and uploaded to the video authentication system for security authentication. When the video is encoded, the video frame includes three types: I frame, B frame and P frame. The compression algorithm is divided into two categories: intra-frame compression and inter-frame compression, where I frame is generated by the former and B and P frames are generated by the latter. The I frame is fully encoded and displays complete information. It is the key frame of the video. The picture can be reconstructed by directly decoding according to its own data. The P frame stands for the forward prediction frame. The forward prediction frame represents a video frame containing the pixel difference between the current frame and the previous frame. The P frame retains the difference information between the current frame and the I frame or P frame before the current frame. When decoding, the corresponding difference information is added on the basis of the reference to the previous video frame to obtain the complete image; the P frame must refer to the forward frame and is also the reference frame of the backward video frame. The accumulation of transmission errors will affect subsequent encoding; the B frame encodes the difference information based on the 1 frame or P frame before and after the current frame. It is also necessary to add the difference information during decoding to obtain the video image. When encoding a video based on a secondary compression algorithm, a complete I frame is first encoded from an adjacent image sequence. If the subsequent video frames are not very different, they are encoded only according to the difference information, thereby obtaining an image sequence consisting of I frames and P frames. This group of image sequences is called a GOP image group. When a frame has a large image difference compared to the previous frame, the previous GOP image group is ended and the next GOP image group is restarted. Therefore, a video may be compressed into multiple GOP image groups for authentication.

[0045] Overlapping video frame groups means that there are M repeated video frames between two adjacent video frame groups. Figure 2 is a schematic diagram of an optional overlapping video frame group according to an embodiment of the present invention, such as Figure 2 As shown, when dividing the video to be authenticated, the video to be authenticated is divided into overlapping video frame groups of length L by selecting an appropriate sliding window. For example, a sliding window of length L=3 is selected, and the sliding window of length 3 is moved backward one frame at a time from the first frame to divide the video into overlapping video frame groups ( Figure 2 Among them, video frame 1, video frame 1 and video frame 3 form SW1, that is, overlapping video frame group 1, video frame 2, video frame 3 and video frame 4 form SW2, that is, overlapping video frame group 2, video frame 4, video frame 5 and video frame 6 form SW4, that is, overlapping video frame group 4), and two adjacent overlapping video frame groups contain two identical video frames.

[0046] Step S102, for each overlapping video frame group, calculate the residual noise of each P frame in the overlapping video frame group, and compare the residual noise of each P frame in the overlapping video frame group with the error value corresponding to the P frame to obtain a first comparison result of the overlapping video frame group.

[0047] It should be noted that in video processing, residual noise refers to the signal remaining after the main image content is removed from the video frame. This usually includes noise components that cannot be removed during image processing or compression. Residual noise can be used to detect whether the video has been edited or tampered with, because the video editing or tampering process will introduce anomalies in the noise characteristics of the image. By monitoring and analyzing the residual noise characteristics, intra-frame forgeries can be identified. Based on the secondary compression algorithm, the residual noise of the P frame is estimated by the noise residual of the video frames adjacent to the P frame. Therefore, by monitoring the noise residual of the P frame, the intra-frame forgery of the video to be authenticated can be quickly identified.

[0048] Optionally, for each overlapping video frame group, the step of calculating the residual noise of each P frame in the overlapping video frame group includes: for each overlapping video frame group, obtaining an I frame in the overlapping video frame group, wherein the I frame is a key frame in the video to be authenticated, and the key frame is a video frame containing complete image data; denoising the I frame based on wavelet transform to obtain a denoised I frame; calculating the difference data between the denoised I frame and the I frame before denoising to obtain the residual noise of the denoised I frame; and calculating the residual noise of the P frame adjacent to the I frame based on the residual noise of the denoised I frame.

[0049] It should be noted that, in the process of calculating the residual noise of the video frame, the I frame in each overlapping video frame group is first determined, the I frame contains complete image data, the I frame is denoised by wavelet transform, and the noise residual of the denoised I frame is calculated, and finally, the P frame adjacent to the I frame is obtained based on the I frame, and the residual noise of the P frame is calculated based on the residual noise of the I frame and the difference information between the P frame and the I frame.

[0050] Optionally, the step of denoising the I frame based on wavelet transform to obtain the denoised I frame includes: decomposing the I frame to obtain grayscale images of the I frame in red RGB channel, green RGB channel and blue RGB channel respectively; performing wavelet transform on the grayscale image of the red RGB channel, the grayscale image of the green RGB channel and the grayscale image of the blue RGB channel of the I frame respectively to obtain a frequency subband set corresponding to the I frame, wherein the frequency subband set includes K frequency subbands of the I frame, and the frequency subband is the frequency component of the I frame in the target direction under the target color channel; for each frequency subband, calculating the local variance of the frequency subband; denoising the frequency subband based on the local variance to obtain the denoised frequency subband, and calculating the wavelet coefficients of the denoised frequency subband; performing inverse wavelet transform on the denoised wavelet coefficients of all frequency subbands to obtain the denoised I frame.

[0051] It should be noted that the embodiment of the present invention denoises the video frame through local variance. The denoising is to remove the noise caused by environmental factors and the like. The noise remaining after denoising is the residual noise. When performing denoising, the video frame is first decomposed into three color channels of red, green and blue, thereby obtaining a grayscale image under the three color channels. Wavelet transform is applied to the grayscale image of each color channel, thereby obtaining multiple frequency sub-bands in the horizontal, vertical and diagonal directions. These frequency sub-bands help separate the different frequency components of the image and decompose the image information into frequency information, which is convenient for more accurate analysis of the image content; the wavelet coefficients of each sub-band are used to calculate the local variance, which is a measure that indicates the degree of variation of pixel values ​​in a specific area. A high local variance usually means that there are more details or noise. After calculating the local variance of each frequency sub-band, the wavelet coefficients of each sub-band can be processed using a Wiener filter. Wiener filtering is a statistical method based on the purpose of minimizing the mean square error between the original signal and the estimated signal, thereby achieving the purpose of denoising. Each frequency sub-band is denoised, and the wavelet coefficients of each sub-band after denoising are calculated, and an inverse wavelet transform is performed on them to obtain a denoised video frame. At the same time, different weights can be assigned to different color channels according to their characteristics, and the overall noise residue of the video frame can be evaluated by the residual noise under each channel and the weight value of the channel.

[0052] Specifically, the calculation formula of local variance can be expressed as: Among them, σ 2 (i,j) is the local variance of the frequency subband, c 2 (i,j) is the high frequency component of the frequency subband, K is a constant, It controls the degree of noise suppression.

[0053] The denoised wavelet coefficients obtained by the Wiener filter are expressed as: Among them, P(i,j) is the wavelet coefficient after denoising.

[0054] Optionally, the step of calculating the residual noise of a P frame adjacent to the I frame based on the residual noise of the denoised I frame includes: for any P frame, selecting an I frame adjacent to the P frame as a reference frame; for each area block in the P frame, obtaining a target area block matching the area block in the P frame in the reference frame; obtaining the pixel difference between the area block in the P frame and the target area block in the reference frame through motion compensation; calculating the residual noise of each area block in the P frame based on the pixel difference between the area block and the target area block and the residual noise of the denoised reference frame on each target area block, so as to obtain the residual noise of the P frame.

[0055] It should be noted that during the video encoding process, according to different video frame types (specifically including I frames and P frames), only the I frame and P frame compression algorithms are used to compare whether there is any forgery in the previous and next video frames. Due to the characteristics of the P frame, when the previous frame is an I frame, the P frame retains the difference between the current frame and the previous I frame, that is, only some areas have noise changes. By calculating the threshold value of the noise change in this part of the area, it can be determined whether there is any intra-frame forgery. For the detection process within the video frame, according to the redundancy between adjacent frames in the video clip, the video frame will be motion compensated after encoding to achieve inter-frame prediction. This feature can be used to calculate the noise residual of the P frame through the noise residual of adjacent video frames. In this process, the region blocks of the video frame can be divided into multiple sizes, such as 16*16 pixel size, 16*8 pixel size, 8*16 pixel size, 8*8 pixel size, 8*4 pixel size, 4*8 pixel size and 4*4 pixel size. The motion compensation of the region blocks of the video frame is performed independently, and all region blocks are obtained by predicting the same region of the reference frame. That is, the P frame only needs to save the coding information of the changed region blocks after the I frame, and the noise of this coding information is different from the coding information of the same region in the I frame.

[0056] Specifically, for any P frame, an I frame adjacent to the P frame is selected as a reference frame. If there is no I frame adjacent to the P frame, the noise residual of the P frame can also be calculated based on the predicted noise residual of the P frame adjacent to the P frame. For each area block in the P frame, a target area block matching the area block in the P frame is obtained in the reference frame, and the pixel difference between the area block in the P frame and the target area block in the reference frame is obtained through motion compensation. Then, based on the pixel difference between the area block and the target area block and the residual noise of the denoised reference frame on each target area block, the residual noise of each area block in the P frame is calculated, and finally the overall residual noise of the P frame is obtained.

[0057] Step S103 , when the first comparison result indicates that the residual noise of all P frames in the overlapping video frame group is less than or equal to the error value corresponding to the P frame, it is determined that there is no intra-frame forgery in the overlapping video frame group.

[0058] It should be noted that by detecting the residual noise of the P frame, forgery in the video frame can be identified. Video editing and tampering will cause the noise characteristics of the video to introduce abnormalities. If the residual noise of the P frame remains within a normal fluctuation range, it indicates that the video is normal and there is no intra-frame forgery. Otherwise, there is intra-frame forgery, which prevents abnormal user terminals from logging in and accessing.

[0059] Optionally, the calculation formula of the error value is expressed as: in, A represents the error value corresponding to the P frame, A represents the error value percentage corresponding to the P frame, S represents the change area block between the P frame and the adjacent video frame, K represents the Kth change area block, K represents the average residual noise in the Kth change area block, and I represents the average residual noise of the P frame.

[0060] It should be noted that the error value can be calculated based on the residual noise of the P frame and the difference between the P frame and the adjacent frames. The error value percentage corresponding to the P frame is calculated based on the noise residual and difference data of the P frame, which is specifically expressed as:

[0061]

[0062] in, represents the error value percentage corresponding to the P frame, S represents the changed area block between the P frame and the adjacent video frame, K represents the Kth changed area block, represents the average residual noise in the Kth change area block, and I represents the average noise of the P frame.

[0063] The noise residual of P frame should be within the error value If the noise of the P frame is detected to be within the range If it is within, it means that there is no forgery in the video frame. Otherwise, it means that forgery occurs in the video frame.

[0064] Optionally, after obtaining the first comparison result of the overlapping video frame group, it also includes: when the first comparison result indicates that the residual noise of any P frame in the overlapping video frame group is greater than the error value, determining that there is intra-video frame forgery in the overlapping video frame; determining that there is an abnormality in the video to be authenticated based on the overlapping video frame with intra-video frame forgery, generating an authentication result of authentication failure for the video to be authenticated with the abnormality, and returning the authentication result to the user end.

[0065] It should be noted that, when the residual noise of any P frame in the overlapping video frame group is greater than the error value, it is determined that there is intra-video frame forgery in the overlapping video frame and it is marked. At the same time, if there is intra-video frame forgery in any overlapping video frame group in the video to be authenticated, it indicates that there is an abnormality in the video to be authenticated, and an authentication result of failed authentication is returned to the user terminal, and login and access of the abnormal user terminal are denied.

[0066] Step S104 , when there is no intra-frame forgery in the N overlapping video frame groups in the video to be authenticated, calculate the similarity value of every two overlapping video frame groups in the video to be authenticated.

[0067] It should be noted that if all overlapping video frame groups in the video to be authenticated do not have any forgery within the video frame, it indicates that there is no abnormality in the video frame to be authenticated, and inter-frame forgery detection continues. Since adjacent frames in the video are continuous in time, the inter-frame forgery operation will change the correlation between adjacent frames of the video. The noise correlation value of adjacent video frames is calculated as the frame group feature, and each overlapping video frame group is used as a query fragment. The similarity value between any two overlapping video frame groups is calculated to perform a similarity check. If the similarity value exceeds the specified threshold, it indicates that the two overlapping video frame groups may be duplicate video fragments that are copied and pasted, that is, there is inter-frame forgery.

[0068] Optionally, the step of calculating the similarity value of every two overlapping video frame groups in the video to be authenticated includes: for each overlapping video frame, combining two adjacent video frames in the overlapping video frame to obtain a video frame pair; calculating the noise correlation value of the video frame pair, and splicing the noise correlation values ​​of all video frame pairs in the overlapping video frame to obtain a noise vector of the overlapping video frame; calculating the similarity value based on the noise vectors of any two overlapping video frames in the video to be authenticated, to obtain the similarity value of every two overlapping video frame groups in the video to be authenticated.

[0069] First, two adjacent video frames in the overlapping video frames are combined to obtain a video frame pair, and the noise correlation value of the video frame pair is calculated. Specifically, the calculation formula of the noise correlation value is expressed as:

[0070]

[0071] Among them, S is the noise correlation value, r represents the rth video frame, and W r (x,y) is the residual noise value of the video frame r at coordinate (x,y), is the average value of the residual noise in video frame r.

[0072] Use e i To represent the correlation between the video frame pairs in the i-th overlapping video frame group, based on the calculation of the noise correlation value, the noise vector of the i-th overlapping video frame group can be obtained.

[0073] Based on the noise vectors of any two overlapping video frames in the video to be authenticated, the similarity value of the two overlapping video frames is calculated. The calculation formula of the similarity value can be expressed as:

[0074]

[0075] Among them, u i and v i is the noise vector and The i-th element of u and γ vis the noise vector and The mean of .

[0076] Step S105 , comparing the similarity value of each two overlapping video frame groups in the video to be authenticated with the similarity threshold to obtain a second comparison result, and determining the authentication result of the video to be authenticated based on the second comparison result.

[0077] Optionally, the step of determining the authentication result of the video to be authenticated based on the second comparison result includes: when the second comparison result indicates that the similarity value of any two overlapping video frame groups in the video to be authenticated is greater than a similarity threshold, determining that there is video frame forgery between the two overlapping video frame groups; determining that there is an abnormality in the video to be authenticated based on the overlapping video frame groups with video frame forgery between them, generating an authentication result of authentication failure for the video to be authenticated with the abnormality, and returning the authentication result to the user end.

[0078] It should be noted that after calculating the similarity values ​​of any two overlapping video frame groups, the similarity values ​​of every two overlapping video frame groups in the video to be authenticated are compared with the similarity threshold. If the similarity values ​​of any two overlapping video frame groups in the video to be authenticated are greater than the similarity threshold, it is determined that there is inter-frame forgery between the two overlapping video frame groups, and thus it is determined that there is an abnormality in the video to be authenticated, and an authentication result of failed authentication is returned to the user terminal, and the login and access of the abnormal user terminal are rejected. On the contrary, if there is no similar overlapping video frame group in the video to be authenticated, it indicates that there is no abnormality in the video to be authenticated, and an authentication result of passed authentication is returned and the login request or access request of the user terminal is approved.

[0079] In this embodiment, the following steps are performed: first, the encoded video to be authenticated is obtained, and the video to be authenticated is divided into N overlapping video frame groups, wherein the video to be authenticated is encoded based on a secondary compression algorithm, and the overlapping video frame group indicates that two adjacent video frame groups contain M repeated video frames, and M and N are both positive integers, and then for each overlapping video frame group, the residual noise of each P frame in the overlapping video frame group is calculated, and the residual noise of each P frame in the overlapping video frame group is compared with the error value corresponding to the P frame, so as to obtain the first comparison result of the overlapping video frame group, wherein the P frame represents a forward prediction frame, and the forward prediction The measured frame represents a video frame containing pixel differences between the current frame and the previous frame, and when the first comparison result indicates that the residual noise of all P frames in the overlapping video frame group is less than or equal to the error value corresponding to the P frame, it is determined that there is no intra-frame forgery in the overlapping video frame group, and when there is no intra-frame forgery in N overlapping video frame groups in the video to be authenticated, the similarity value of each two overlapping video frame groups in the video to be authenticated is calculated, and finally the similarity value of each two overlapping video frame groups in the video to be authenticated is compared with the similarity threshold to obtain a second comparison result, and the authentication result of the video to be authenticated is determined based on the second comparison result.

[0080] In this embodiment, when authenticating a video, the video to be authenticated is divided into N overlapping video frame groups, and the overlapping video frame groups are used as detection objects. The residual noise of the P frames in the group is calculated and compared with the preset error value, so that the tampering in the frame can be detected accurately. At the same time, the similarity value between the video frame groups is calculated and compared with the set similarity threshold, so that the tampering between frames can be detected. There is no need to detect the video frame by frame, which can effectively reduce the amount of calculation, improve the detection efficiency, and maintain high-precision forgery detection within and between frames. This solves the technical problem in the related art that when authenticating a video, the video frames in the video need to be detected frame by frame, and the detection efficiency is low.

[0081] The embodiments of the present invention are described in more detail below with reference to specific examples.

[0082] Example 1:

[0083] During a user login process, an authentication video is obtained, which involves the user's head shaking action; the video device used to record the authentication video digitizes the recorded authentication video (such as the common H.264 encoding). The main purpose of encoding is to reduce the size of video data for easy storage and transmission. The authentication video is detected by the above authentication method. The specific steps include:

[0084] Step 1, according to the received coded video, which may be in the form of IP-...-P frames, the coded video is divided into a plurality of overlapping video frame groups by dividing the video frame groups; the divided video frame groups are used for detecting between subsequent video frames;

[0085] Step 2, extract noise from the video frame I frame, use wavelet transform through RGB color channels to extract noise features on each channel, and realize noise extraction of the video frame through local variance calculation and Wiener filter denoising. The noise of the video frame P frame is synthesized according to the noise change of the changed regional block; in the secondary compression algorithm, calculate the error value of the regional block noise change between the I frame and the P frame, and judge whether forgery has occurred in the frame according to whether the noise change is within the error value. Specifically: if in the video frames where the I frame and the P frame are continuous, the I frame is the front image of the user, and the P frame is the middle image of the shaking head. During the detection process, the deflection angle of the middle image of the shaking head is too large. Then, during the noise extraction process, the error value will be too large (for example, the video frame is divided into 0.0833s, and the shaking head deflection angle is normally 0-15°). Here, it can be considered that forgery has occurred in the frame.

[0086] Step 3, if there is no forgery in the video frame, several video frames are grouped together and their video frame noise is merged to calculate the similarity between the video frame groups, and the connection between the video frames is judged by the similarity. In the continuous video frames, the previous video frame is associated with the current video frame and the next video frame. In this embodiment, three video frames are grouped together. The secondary compression algorithm has been used to detect that there is no intra-frame forgery in the first video frame group. The first video frame group is used as a unit, and the subsequent video frames are detected in the form of video frame groups. Since adjacent frames in the video are continuous in time, the forgery operation between video frames will change the correlation between adjacent frames of the video. The adjacent noise correlation is extracted as a frame group feature representation, and each overlapping video frame group is used as a query segment. The similarity value between the overlapping video frame groups is calculated to perform a similarity check. If the similarity exceeds the specified threshold, the corresponding subgroup is selected as a forged group candidate, and the feature continuity of the adjacent frames is analyzed to determine whether there is a forged video frame. Specifically: in the above authentication video of the head shaking action, incoherent actions appear in the P5 and P6 frames, and the correlation between the video frame group 5 and the video frame group 6 will be greatly different, so that P6 is detected as an inserted video frame.

[0087] Step 4, in order to determine the similarity between the obtained overlapping video frame groups, the similarity between each group and all other groups is calculated, and these values ​​are combined to form an n*n dimensional correlation matrix C. Therefore, in each row of the matrix C, the similarity value between the frame group and other frame groups is maintained. After calculating the similarity values ​​between all overlapping video frame groups, the matrix C is further simplified to obtain the final matrix; if the similarity value is greater than the specified similarity threshold tha, the two related video frame groups are regarded as repeated forgeries. This step is to detect another situation of synthetic authentication video, that is, the copy and paste of video frames. Specifically: before and after the authentication video, if the similarity of the video frame group is 1, it is judged to be a copy and paste of the video frame. The situation where the similarity is 1 is: the video frames in the frame group are completely consistent.

[0088] Example 2:

[0089] In the application scenario of digital campus, a video authentication system is adopted to realize the security detection of the authentication video. The video authentication system includes video equipment, data center and detection center. The video equipment is equivalent to the equipment used by users to log in, such as cameras of electronic products such as mobile phones and computers, which are used to shoot authentication videos. The data center can be composed of login systems of various campus platforms, which implements the issuance of authentication video shooting instructions through instruction processing, and receives authentication videos uploaded by video equipment at the same time, and transmits them to the detection center. The detection center is used to implement the above-mentioned digital campus platform security authentication method based on image processing, and realizes the detection of video authenticity by performing false video recognition on the authentication video. The detection content includes intra-frame detection and inter-frame detection. The intra-frame detection is mainly used to detect whether AI face-changing occurs, and the inter-frame detection is used to detect whether video synthesis occurs.

[0090] When students, parents or staff log in through video authentication based on image processing, the specific process is as follows:

[0091] Video recording: After the user chooses to log in using video authentication, the data center will ask the user to record a short video. This video may include asking the user to say a specific password or perform specific actions.

[0092] Video upload: The recorded video will be uploaded to the data center and transmitted to the testing center for processing.

[0093] Image processing: The detection center uses image processing technology, including video compression algorithm, wavelet transform, noise extraction and correlation calculation to process the video.

[0094] Noise extraction: Perform wavelet transform on the video frame to extract the noise information of the video frame.

[0095] Secondary compression algorithm: In the secondary compression algorithm, since the noise of the P frame can be estimated from the I frame in the GOP image group, the error value between the I frame and the P frame is used to determine whether there is forgery between the frames;

[0096] Correlation calculation: Calculate the similarity between video frame groups to determine whether synthesis occurs between video frames.

[0097] Login permission: If the video passes the authenticity verification, the user will be authorized to log in to the campus platform to perform related operations. Otherwise, the testing center will issue an alarm and restrict the login of the video device.

[0098] The entire authentication process uses image processing technology to ensure the authenticity of the video. The system checks the noise information of the video and the correlation between the video frame groups to determine the validity of the video. This authentication process can make the identity authentication of students, parents and staff using video to log in more secure and reliable, thereby ensuring the security of the campus platform.

[0099] The following is a detailed description in conjunction with another embodiment.

[0100] Embodiment 2

[0101] A video authentication device based on image processing provided in this embodiment includes multiple implementation units, each of which corresponds to each implementation step in the above-mentioned embodiment 1. Its specific implementation methods and beneficial effects can refer to the above-mentioned method embodiments and will not be repeated here.

[0102] Figure 3 is a schematic diagram of an optional video authentication device based on image processing according to an embodiment of the present invention, such as Figure 3 As shown, the video authentication device based on image processing may include: an acquisition unit 31, a first calculation unit 32, a determination unit 33, a second calculation unit 34, and a comparison unit 35, wherein:

[0103] The acquisition unit 31 is used to acquire the encoded video to be authenticated, and divide the video to be authenticated into N overlapping video frame groups, wherein the video to be authenticated is encoded based on a secondary compression algorithm, and the overlapping video frame group means that there are M repeated video frames between two adjacent video frame groups, and M and N are both positive integers;

[0104] The first calculation unit 32 is used to calculate the residual noise of each P frame in the overlapping video frame group for each overlapping video frame group, and compare the residual noise of each P frame in the overlapping video frame group with the error value corresponding to the P frame to obtain a first comparison result of the overlapping video frame group, wherein the P frame represents a forward prediction frame, and the forward prediction frame represents a video frame containing pixel differences between a current frame and a previous frame;

[0105] A determination unit 33, configured to determine that there is no intra-frame forgery in the overlapping video frame group when the first comparison result indicates that the residual noise of all P frames in the overlapping video frame group is less than or equal to the error value corresponding to the P frame;

[0106] The second calculation unit 34 is used to calculate the similarity value of every two overlapping video frame groups in the video to be authenticated when there is no intra-frame forgery in the N overlapping video frame groups in the video to be authenticated;

[0107] The comparison unit 35 is used to compare the similarity value of each two overlapping video frame groups in the video to be authenticated with the similarity threshold to obtain a second comparison result, and determine the authentication result of the video to be authenticated based on the second comparison result.

[0108] The above-mentioned video authentication device based on image processing obtains the encoded video to be authenticated through the acquisition unit 31, and divides the video to be authenticated into N overlapping video frame groups, wherein the video to be authenticated is encoded based on a secondary compression algorithm, and the overlapping video frame group indicates that there are M repeated video frames between two adjacent video frame groups, and M and N are both positive integers; for each overlapping video frame group, the first calculation unit 32 calculates the residual noise of each P frame in the overlapping video frame group, and compares the residual noise of each P frame in the overlapping video frame group with the error value corresponding to the P frame, so as to obtain the first comparison result of the overlapping video frame group, wherein the P frame represents a forward prediction frame, and the forward prediction frame represents a packet A video frame containing pixel differences between a current frame and a previous frame; determining that there is no intra-frame forgery in the overlapping video frame group by a determining unit 33 when a first comparison result indicates that the residual noise of all P frames in the overlapping video frame group is less than or equal to the error value corresponding to the P frame; calculating the similarity value of each two overlapping video frame groups in the video to be authenticated by a second calculating unit 34 when there is no intra-frame forgery in N overlapping video frame groups in the video to be authenticated; a comparing unit 35, comparing the similarity value of each two overlapping video frame groups in the video to be authenticated with a similarity threshold to obtain a second comparison result, and determining the authentication result of the video to be authenticated based on the second comparison result.

[0109] In this embodiment, when authenticating a video, the video to be authenticated is divided into N overlapping video frame groups, and the overlapping video frame groups are used as detection objects. The residual noise of the P frames in the group is calculated and compared with the preset error value, so that the tampering in the frame can be detected accurately. At the same time, the similarity value between the video frame groups is calculated and compared with the set similarity threshold, so that the tampering between frames can be detected. There is no need to detect the video frame by frame, which can effectively reduce the amount of calculation, improve the detection efficiency, and maintain high-precision forgery detection within and between frames. This solves the technical problem in the related art that when authenticating a video, the video frames in the video need to be detected frame by frame, and the detection efficiency is low.

[0110] Optionally, the first calculation unit 32 includes: a first acquisition module, used to obtain an I frame in the overlapping video frame group for each overlapping video frame group, wherein the I frame is a key frame in the video to be authenticated, and the key frame is a video frame containing complete image data; a first denoising module, used to denoise the I frame based on wavelet transform to obtain a denoised I frame; a first calculation module, used to calculate the difference data between the denoised I frame and the I frame before denoising to obtain the residual noise of the denoised I frame; and a second calculation module, used to calculate the residual noise of the P frame adjacent to the I frame based on the residual noise of the denoised I frame.

[0111] Optionally, the first denoising module also includes: a first decomposition submodule, used to decompose the I frame to obtain grayscale images of the I frame in the red RGB channel, the green RGB channel and the blue RGB channel respectively; a first transformation submodule, used to perform wavelet transform on the grayscale image of the red RGB channel, the grayscale image of the green RGB channel and the grayscale image of the blue RGB channel of the I frame respectively to obtain a frequency subband set corresponding to the I frame, wherein the frequency subband set includes K frequency subbands of the I frame, and the frequency subband is the frequency component of the I frame in the target direction under the target color channel; a first calculation submodule, used to calculate the local variance of the frequency subband for each frequency subband; a first denoising submodule, used to perform denoising on the frequency subband based on the local variance to obtain the denoised frequency subband, and calculate the wavelet coefficients of the denoised frequency subband; a second transformation submodule, used to perform inverse wavelet transform on the denoised wavelet coefficients of all frequency subbands to obtain the denoised I frame.

[0112] Optionally, the first calculation module includes: a first selection submodule, used for selecting an I frame adjacent to the P frame as a reference frame for any P frame; a first acquisition submodule, used for acquiring, for each area block in the P frame, a target area block that matches the area block in the P frame in the reference frame; a second acquisition submodule, used for acquiring the pixel difference between the area block in the P frame and the target area block in the reference frame through motion compensation; and a second calculation submodule, used for calculating the residual noise of each area block in the P frame based on the pixel difference between the area block and the target area block and the residual noise of the denoised reference frame on each target area block, so as to obtain the residual noise of the P frame.

[0113] Optionally, the second calculation unit 34 includes: a first combination module, used for combining two adjacent video frames in the overlapping video frames for each overlapping video frame to obtain a video frame pair; a third calculation module, used for calculating the noise correlation value of the video frame pair, and splicing the noise correlation values ​​of all video frame pairs in the overlapping video frames to obtain a noise vector of the overlapping video frame; and a fourth calculation module, used for calculating the similarity value based on the noise vectors of any two overlapping video frames in the video to be authenticated, and obtaining the similarity value of each two overlapping video frame groups in the video to be authenticated.

[0114] Optionally, the video authentication device based on image processing also includes: a first determination module, used to determine that there is intra-video frame forgery in the overlapping video frame when the first comparison result indicates that the residual noise of any P frame in the overlapping video frame group is greater than the error value; a first generation module, used to determine that there is an abnormality in the video to be authenticated based on the overlapping video frame with intra-video frame forgery, generate an authentication result of authentication failure for the video to be authenticated with the abnormality, and return the authentication result to the user end.

[0115] Optionally, the determination unit 33 includes: a second determination module, used to determine that there is video frame forgery between two overlapping video frame groups when the second comparison result indicates that the similarity value of any two overlapping video frame groups in the video to be authenticated is greater than a similarity threshold; a third determination module, used to determine that there is an abnormality in the video to be authenticated based on the overlapping video frame groups with video frame forgery between them, generate an authentication result of authentication failure for the video to be authenticated with the abnormality, and return the authentication result to the user end.

[0116] Optionally, the calculation formula of the error value is expressed as: in, A represents the error value corresponding to the P frame, represents the error value percentage corresponding to the P frame, S represents the changed area block between the P frame and the adjacent video frame, K represents the Kth changed area block, represents the average residual noise in the Kth change area block, and I represents the average residual noise in the P frame.

[0117] The above-mentioned video authentication device based on image processing may also include a processor and a memory. The above-mentioned acquisition unit 31, the first calculation unit 32, the determination unit 33, the second calculation unit 34, the comparison unit 35, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions.

[0118] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the security authentication of the video can be performed by adjusting the kernel parameters.

[0119] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (fl ash RAM), and the memory includes at least one storage chip.

[0120] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, the computer-readable storage medium including a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned video authentication methods based on image processing.

[0121] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement any one of the above-mentioned video authentication methods based on image processing.

[0122] According to another aspect of an embodiment of the present invention, a computer program product is further provided. The computer program product includes a computer program, wherein when the computer program is executed by a processor, any one of the above-mentioned video authentication methods based on image processing is implemented.

[0123] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that is initialized with the following method steps: obtaining the encoded video to be authenticated, and dividing the video to be authenticated into N overlapping video frame groups, wherein the video to be authenticated is encoded based on a secondary compression algorithm, and the overlapping video frame group indicates that there are M repeated video frames between two adjacent video frame groups, and M and N are both positive integers; for each overlapping video frame group, calculating the residual noise of each P frame in the overlapping video frame group, and comparing the residual noise of each P frame in the overlapping video frame group with the error value corresponding to the P frame, to obtain a first comparison result of the overlapping video frame group , wherein the P frame represents a forward prediction frame, and the forward prediction frame represents a video frame containing the pixel difference between the current frame and the previous frame; when the first comparison result indicates that the residual noise of all P frames in the overlapping video frame group is less than or equal to the error value corresponding to the P frame, it is determined that there is no intra-frame forgery in the overlapping video frame group; when there is no intra-frame forgery in the N overlapping video frame groups in the video to be authenticated, the similarity value of each two overlapping video frame groups in the video to be authenticated is calculated; the similarity value of each two overlapping video frame groups in the video to be authenticated is compared with the similarity threshold to obtain a second comparison result, and the authentication result of the video to be authenticated is determined based on the second comparison result.

[0124] The present application also provides a computer program product, which, when executed on a data processing device, is also suitable for executing a program initialized with the following method steps: for each overlapping video frame group, the step of calculating the residual noise of each P frame in the overlapping video frame group includes: for each overlapping video frame group, obtaining an I frame in the overlapping video frame group, wherein the I frame is a key frame in the video to be authenticated, and the key frame is a video frame containing complete image data; denoising the I frame based on wavelet transform to obtain a denoised I frame; calculating the difference data between the denoised I frame and the I frame before denoising to obtain the residual noise of the denoised I frame; and calculating the residual noise of the P frame adjacent to the I frame based on the residual noise of the denoised I frame.

[0125] The present application also provides a computer program product, which, when executed on a data processing device, is also suitable for executing a program that is initialized with the following method steps: denoising an I frame based on a wavelet transform, and the step of obtaining a denoised I frame includes: decomposing the I frame to obtain grayscale images of the I frame in a red RGB channel, a green RGB channel and a blue RGB channel respectively; performing a wavelet transform on the grayscale image of the red RGB channel, the grayscale image of the green RGB channel and the grayscale image of the blue RGB channel of the I frame respectively to obtain a frequency subband set corresponding to the I frame, wherein the frequency subband set includes K frequency subbands of the I frame, and the frequency subband is a frequency component of the I frame in a target direction under a target color channel; for each frequency subband, calculating the local variance of the frequency subband; denoising the frequency subband based on the local variance to obtain a denoised frequency subband, and calculating the wavelet coefficients of the denoised frequency subband; performing an inverse wavelet transform on the denoised wavelet coefficients of all frequency subbands to obtain a denoised I frame.

[0126] The present application also provides a computer program product, which, when executed on a data processing device, is also suitable for executing a program that is initialized with the following method steps: the step of calculating the residual noise of a P frame adjacent to the I frame based on the residual noise of the I frame after denoising includes: for any P frame, selecting the I frame adjacent to the P frame as a reference frame; for each area block in the P frame, obtaining a target area block that matches the area block in the P frame in the reference frame; obtaining the pixel difference between the area block in the P frame and the target area block in the reference frame through motion compensation; calculating the residual noise of each area block in the P frame based on the pixel difference between the area block and the target area block and the residual noise of the reference frame after denoising on each target area block, so as to obtain the residual noise of the P frame.

[0127] The present application also provides a computer program product, which, when executed on a data processing device, is also suitable for executing a program initialized with the following method steps: the step of calculating the similarity value of each two overlapping video frame groups in the video to be authenticated includes: for each overlapping video frame, combining two adjacent video frames in the overlapping video frame to obtain a video frame pair; calculating the noise correlation value of the video frame pair, and splicing the noise correlation values ​​of all video frame pairs in the overlapping video frame to obtain a noise vector of the overlapping video frame; calculating the similarity value based on the noise vectors of any two overlapping video frames in the video to be authenticated, and obtaining the similarity value of each two overlapping video frame groups in the video to be authenticated.

[0128] The present application also provides a computer program product, which, when executed on a data processing device, is also suitable for executing a program that is initialized with the following method steps: after obtaining a first comparison result of the overlapping video frame group, it also includes: when the first comparison result indicates that the residual noise of any P frame in the overlapping video frame group is greater than an error value, determining that there is intra-video frame forgery in the overlapping video frame; determining that there is an abnormality in the video to be authenticated based on the overlapping video frame with intra-video frame forgery, generating an authentication result of authentication failure for the video to be authenticated with the abnormality, and returning the authentication result to the user end.

[0129] The present application also provides a computer program product, which, when executed on a data processing device, is also suitable for executing a program initialized with the following method steps: the step of determining the authentication result of the video to be authenticated based on the second comparison result includes: when the second comparison result indicates that the similarity value of any two overlapping video frame groups in the video to be authenticated is greater than a similarity threshold, determining that there is inter-frame forgery of video frames between the two overlapping video frame groups; determining that there is an abnormality in the video to be authenticated based on the overlapping video frame groups with inter-frame forgery of video frames, generating an authentication result of authentication failure for the video to be authenticated with the abnormality, and returning the authentication result to the user end.

[0130] The present application also provides a computer program product, which, when executed on a data processing device, is also suitable for executing a program for initializing the following method steps: the calculation formula of the error value is expressed as: in, A represents the error value corresponding to the P frame, represents the error value percentage corresponding to the P frame, S represents the change area block between the P frame and the adjacent video frame, K represents the Kth change area block, K represents the average residual noise in the Kth change area block, and I represents the average residual noise of the P frame.

[0131] Figure 4 1 is a hardware structure block diagram of an electronic device (or mobile device) according to an embodiment of the present invention, which is a video authentication method based on image processing. Figure 4 As shown, the electronic device may include one or more ( Figure 4 402a, 402b, ..., 402n are used to illustrate) processor 402 (processor 402 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), memory 404 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 4 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 4 More or fewer components as shown, or with Figure 4 Different configurations shown.

[0132] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0133] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0134] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0135] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0136] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0137] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0138] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A video authentication method based on image processing, characterized in that: include: Obtaining the encoded video to be authenticated, and dividing the video to be authenticated into N overlapping video frame groups, wherein the video to be authenticated is encoded based on a secondary compression algorithm, and the overlapping video frame group means that there are M repeated video frames between two adjacent video frame groups, and M and N are both positive integers; For each of the overlapping video frame groups, the residual noise of each P frame in the overlapping video frame group is calculated, and the residual noise of each P frame in the overlapping video frame group is compared with the error value corresponding to the P frame to obtain a first comparison result of the overlapping video frame group, wherein the P frame represents a forward prediction frame, and the forward prediction frame represents a video frame containing pixel differences between a current frame and a previous frame; When the first comparison result indicates that the residual noise of all P frames in the overlapping video frame group is less than or equal to the error value corresponding to the P frame, determining that there is no intra-frame forgery in the overlapping video frame group; In the case that there is no intra-frame forgery in the N overlapping video frame groups in the video to be authenticated, calculating the similarity value of every two overlapping video frame groups in the video to be authenticated; The similarity value of each two overlapping video frame groups in the video to be authenticated is compared with a similarity threshold to obtain a second comparison result, and the authentication result of the video to be authenticated is determined based on the second comparison result.

2. The video authentication method according to claim 1, characterized in that: For each of the overlapping video frame groups, the step of calculating the residual noise of each P frame in the overlapping video frame group comprises: For each of the overlapping video frame groups, obtaining an I frame in the overlapping video frame group, wherein the I frame is a key frame in the video to be authenticated, and the key frame is a video frame containing complete image data; De-noising the I frame based on wavelet transform to obtain a denoised I frame; Calculate the difference data between the I frame after denoising and the I frame before denoising to obtain the residual noise of the I frame after denoising; The residual noise of the P frame adjacent to the I frame is calculated based on the residual noise of the denoised I frame.

3. The video authentication method according to claim 2, characterized in that: The steps of denoising the I frame based on wavelet transform to obtain the denoised I frame include: Decompose the I frame to obtain grayscale images of the I frame in the red RGB channel, the green RGB channel, and the blue RGB channel; Performing wavelet transform on the grayscale image of the red RGB channel, the grayscale image of the green RGB channel, and the grayscale image of the blue RGB channel of the I frame respectively to obtain a frequency subband set corresponding to the I frame, wherein the frequency subband set includes K frequency subbands of the I frame, and the frequency subbands are frequency components of the I frame in a target direction under a target color channel; For each of the frequency sub-bands, calculating the local variance of the frequency sub-band; Performing denoising processing on the frequency sub-band based on the local variance to obtain a denoised frequency sub-band, and calculating a wavelet coefficient of the denoised frequency sub-band; An inverse wavelet transform is performed on the denoised wavelet coefficients of all the frequency sub-bands to obtain a denoised I frame.

4. The video authentication method according to claim 2, characterized in that: The step of calculating the residual noise of the P frame adjacent to the I frame based on the residual noise of the denoised I frame comprises: For any P frame, select the I frame adjacent to the P frame as the reference frame; For each region block in the P frame, a target region block matching the region block in the P frame is obtained in the reference frame; Obtain pixel differences between a region block in a P frame and a target region block in a reference frame through motion compensation; The residual noise of each region block in the P frame is calculated based on the pixel difference between the region block and the target region block and the residual noise of the denoised reference frame on each target region block to obtain the residual noise of the P frame.

5. The video authentication method according to claim 1, characterized in that: The step of calculating the similarity value of each two overlapping video frame groups in the video to be authenticated comprises: For each of the overlapping video frames, combining two adjacent video frames in the overlapping video frames to obtain a video frame pair; Calculating the noise correlation value of the video frame pair, and concatenating the noise correlation values ​​of all video frame pairs in the overlapping video frame to obtain a noise vector of the overlapping video frame; The similarity value is calculated based on the noise vectors of any two overlapping video frames in the video to be authenticated, so as to obtain the similarity value of every two overlapping video frame groups in the video to be authenticated.

6. The video authentication method according to claim 1, characterized in that: After obtaining the first comparison result of the overlapping video frame group, the method further includes: When the first comparison result indicates that the residual noise of any P frame in the overlapping video frame group is greater than the error value, determining that there is intra-video frame forgery in the overlapping video frame; Based on the forged overlapping video frames in the existing video frames, it is determined that the video to be authenticated has an abnormality, an authentication result indicating authentication failure is generated for the video to be authenticated with the abnormality, and the authentication result is returned to the user end.

7. The video authentication method according to claim 1, characterized in that: The step of determining the authentication result of the video to be authenticated based on the second comparison result includes: When the second comparison result indicates that the similarity value of any two overlapping video frame groups in the video to be authenticated is greater than a similarity threshold, determining that there is inter-frame forgery of video frames between the two overlapping video frame groups; Based on the overlapping video frame group with inter-frame forgery, it is determined that the video to be authenticated has an abnormality, an authentication result indicating authentication failure is generated for the video to be authenticated with the abnormality, and the authentication result is returned to the user end.

8. The video authentication method according to claim 1, characterized in that: The calculation formula of the error value is expressed as: in, A represents the error value corresponding to the P frame, represents the error value percentage corresponding to the P frame, S represents the changed area block between the P frame and the adjacent video frame, K represents the Kth changed area block, represents the average residual noise in the Kth change area block, and I represents the average residual noise in the P frame.

9. A video authentication device based on image processing, characterized in that: include: An acquisition unit is used to acquire the encoded video to be authenticated, and divide the video to be authenticated into N overlapping video frame groups, wherein the video to be authenticated is encoded based on a secondary compression algorithm, and the overlapping video frame group means that there are M repeated video frames between two adjacent video frame groups, and M and N are both positive integers; A first calculation unit is used to calculate the residual noise of each P frame in the overlapping video frame group for each overlapping video frame group, and compare the residual noise of each P frame in the overlapping video frame group with the error value corresponding to the P frame to obtain a first comparison result of the overlapping video frame group, wherein the P frame represents a forward prediction frame, and the forward prediction frame represents a video frame containing a pixel difference between a current frame and a previous frame; a determining unit, configured to determine that there is no intra-frame forgery in the overlapping video frame group when the first comparison result indicates that the residual noise of all P frames in the overlapping video frame group is less than or equal to the error value corresponding to the P frame; A second calculation unit is used to calculate the similarity value of every two overlapping video frame groups in the video to be authenticated when there is no intra-frame forgery in the N overlapping video frame groups in the video to be authenticated; The comparison unit is used to compare the similarity value of each two overlapping video frame groups in the video to be authenticated with a similarity threshold to obtain a second comparison result, and determine the authentication result of the video to be authenticated based on the second comparison result.

10. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the video authentication method based on image processing as described in any one of claims 1 to 8.

Images