Video detection method and device, equipment and storage medium

Abstract

The embodiment of the invention discloses a video detection method and device, equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the steps of obtaining a to-be-detected frame sequence of a to-be-detected video; the to-be-detected frame sequence is input into a trained video detection model, a target prediction frame sequence is output, and the trained video detection model is obtained through training on the basis of attention features between video frames of a training video; obtaining the similarity between the target prediction frame sequence and the to-be-detected frame sequence; and comparing the similarity with a preset similarity threshold, when the similarity is greater than the preset similarity threshold, determining that the to-be-detected video is an abnormal video, and when the similarity is not greater than the preset similarity threshold, determining that the to-be-detected video is a normal video. Therefore, the similarity between the to-be-detected frame sequence and the target prediction frame sequence is compared with the preset similarity threshold, so that whether the to-be-detected video is an abnormal video can be accurately detected.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a video detection method, device, equipment and storage medium. Background Art

[0002] With the rapid development of the Internet and the popularization of smart devices, the use and transmission of video data has become an indispensable part of our daily life. However, with it comes a large amount of video data that needs to be managed and processed. In order to improve the quality and security of video data, video anomaly detection technology came into being.

[0003] In some application scenarios, it is necessary to identify faces through videos to achieve permission verification. In related technologies, permission verification is often performed by simple facial feature detection and then matching based on facial features. Assuming that the video used for facial feature detection is generated by obtaining a real face and then using a computer device to virtually generate it, the detection mechanism in the related technology cannot detect the authenticity of the video itself, and only considers the detection of facial features in the video. In this case, false videos may bypass the facial detection mechanism in the related technology, resulting in unreasonable authorization and ultimately causing property losses to users.

[0004] Therefore, the related technology cannot detect the authenticity of the video itself, and can only detect the content in the video. Summary of the invention

[0005] The main purpose of the embodiments of the present application is to provide a video detection method, device, equipment and storage medium, which can accurately detect whether the video to be detected is abnormal.

[0006] To achieve the above object, a first aspect of an embodiment of the present application proposes a video detection method, comprising:

[0007] Obtain a frame sequence to be detected of a video to be detected;

[0008] Inputting the frame sequence to be detected into a trained video detection model, and outputting a target prediction frame sequence, wherein the trained video detection model is trained based on attention features between video frames of a training video;

[0009] Obtaining the similarity between the target prediction frame sequence and the frame sequence to be detected;

[0010] Comparing the similarity with a preset similarity threshold, and when the similarity is greater than the preset similarity threshold, determining that the video to be detected is an abnormal video;

[0011] When the similarity is not greater than the preset similarity threshold, it is determined that the video to be detected is a normal video.

[0012] In some implementations, the obtaining the similarity between the target prediction frame sequence and the to-be-detected frame sequence includes:

[0013] Determining a target prediction frame in the target prediction frame sequence, and determining a target video frame corresponding to each target prediction frame in the to-be-detected frame sequence;

[0014] Determining a frame similarity between each target prediction frame and a target video frame corresponding to each target prediction frame;

[0015] The similarity between the target prediction frame sequence and the to-be-detected frame sequence is determined according to the frame similarity.

[0016] In some implementations, determining the similarity between the target prediction frame sequence and the to-be-detected frame sequence according to the frame similarity includes:

[0017] Determining the number of frames of the target prediction frame sequence;

[0018] The frame similarities corresponding to each target prediction frame are added and divided by the number of frames to obtain the similarity between the target prediction frame sequence and the frame sequence to be detected.

[0019] In some embodiments, the video detection model includes an encoding sub-model, an attention sub-model, and a decoding sub-model, and before inputting the frame sequence to be detected into the trained video detection model and outputting the target prediction frame sequence, it also includes:

[0020] Get a video frame sequence of a training video;

[0021] Inputting the video frame sequence into the coding sub-model, and outputting the coding features corresponding to each video frame in the video frame sequence;

[0022] Inputting the encoded features into the attention sub-model to obtain attention features between each video frame and other video frames;

[0023] Inputting the attention features into the decoding sub-model and outputting a predicted video frame sequence;

[0024] Determine the loss value between the predicted video frame sequence and the video frame sequence. When the loss value is greater than a preset loss value, adjust the model parameters of the video detection model according to the loss value to implement iteration of the video detection model, and return to execute the video frame sequence of the training video until the loss value is no greater than the preset loss value, thereby obtaining a trained video detection model.

[0025] In some embodiments, the encoding sub-model includes multiple convolutional layers, multiple pooling layers and a fully connected layer, the output of the previous convolutional layer is connected to the input of the next pooling layer, the output of the next pooling layer is connected to the input of the next convolutional layer, and the last pooling layer is connected to the fully connected layer;

[0026] The step of inputting the video frame sequence into the coding sub-model and outputting coding features corresponding to each video frame in the video frame sequence includes:

[0027] Input the video frame sequence from the first convolutional layer of the encoding sub-model, and obtain the video frame sequence features output by the fully connected layer of the encoding sub-model;

[0028] Perform dimensionality reduction processing on the features of the video frame sequence, and output the encoding features corresponding to each video frame in the video frame sequence.

[0029] In some embodiments, the attention sub-model includes a plurality of attention modules, and inputting the encoded features into the attention sub-model to obtain the attention features between each video frame and other video frames includes:

[0030] Input each of the encoded features into the first attention module to obtain a sub-attention feature output by the first attention module;

[0031] Inputting the sub-attention feature output by the first attention module into the next attention module to obtain the sub-attention feature output by the next attention module;

[0032] Repeatedly input the sub-attention features output by the previous attention module into the next attention module until the last attention module outputs the attention features corresponding to each of the encoded features, where the attention features are the attention features between each video frame relative to other video frames.

[0033] In some embodiments, the decoding sub-model includes a plurality of sequentially connected deconvolution layers and a fully connected layer, and the last deconvolution layer of the plurality of sequentially connected deconvolution layers is connected to the fully connected layer;

[0034] The step of inputting the attention feature into the decoding sub-model and outputting a predicted video frame sequence comprises:

[0035] Inputting the attention features into a plurality of sequentially connected deconvolution layers to obtain decoding features corresponding to each video frame;

[0036] The decoded features are input into the fully connected layer, and a predicted video frame sequence is output.

[0037] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a video detection device, comprising:

[0038] A first acquisition module is used to acquire a frame sequence to be detected of a video to be detected;

[0039] An input module, used to input the frame sequence to be detected into a trained video detection model, and output a target prediction frame sequence, wherein the trained video detection model is trained based on attention features between video frames of a training video;

[0040] A second acquisition module, used to acquire the similarity between the target prediction frame sequence and the to-be-detected frame sequence;

[0041] A first determination module is used to compare the similarity with a preset similarity threshold, and when the similarity is greater than the preset similarity threshold, determine that the video to be detected is an abnormal video;

[0042] The second determination module is used to determine that the video to be detected is a normal video when the similarity is not greater than the preset similarity threshold.

[0043] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes a computer-readable storage medium, which stores a plurality of instructions suitable for loading by a processor to execute the video detection method provided in the embodiment of the present application.

[0044] To achieve the above-mentioned objectives, the fourth aspect of an embodiment of the present application proposes a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the video detection method provided in the embodiment of the present application when executing the computer program.

[0045] In an embodiment of the present application, by obtaining a frame sequence to be detected of a video to be detected; inputting the frame sequence to be detected into a trained video detection model, outputting a target prediction frame sequence, the trained video detection model is obtained based on the attention features between the video frames of the training video; obtaining the similarity between the target prediction frame sequence and the frame sequence to be detected; comparing the similarity with a preset similarity threshold, when the similarity is greater than the preset similarity threshold, determining that the video to be detected is an abnormal video, and when the similarity is not greater than the preset similarity threshold, determining that the video to be detected is a normal video. In this way, through the attention mechanism in the trained video detection model, the feature extraction of the frame sequence to be detected of the video to be detected can be accurately performed, and then the extracted features are used to predict the target prediction frame sequence corresponding to the frame sequence to be detected, and finally by comparing the similarity between the frame sequence to be detected and the target prediction frame sequence, and comparing the similarity with the preset similarity threshold, it is possible to accurately detect whether the video to be detected is an abnormal video.

[0046] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 is a schematic diagram of a scene of video detection provided in an embodiment of the present application;

[0049] Figure 2 It is a flowchart of a video detection method provided in an embodiment of the present application;

[0050] Figure 3 It is a schematic diagram of the training process of the video detection model provided in the embodiment of the present application;

[0051] Figure 4 is a schematic diagram of the structure of the video detection model provided in the embodiment of the present application;

[0052] Figure 5 It is a schematic diagram of the structure of the encoding sub-model provided in the embodiment of the present application;

[0053] Figure 6 is a schematic diagram of the structure of the attention sub-model provided in an embodiment of the present application;

[0054] Figure 7 It is a schematic diagram of the structure of the decoding sub-model provided in the embodiment of the present application;

[0055] Figure 8 is another flow chart of the video detection method provided in an embodiment of the present application;

[0056] Fig. 9 is a structural schematic diagram of a video detection device provided in an embodiment of the present application;

[0057] Fig.10 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0059] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0061] First, some nouns involved in this application are analyzed:

[0062] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0063] Self-Attention Mechanism: The self-attention mechanism mainly allows the model to learn to pay attention to the information at different locations within itself, thereby automatically assigning different importance weights to different parts. Unlike the traditional attention mechanism, the self-attention mechanism does not rely on external information sources, but calculates the attention weights entirely based on the input data itself. In tasks such as image classification, object detection, and image segmentation, the self-attention mechanism allows the model to focus on different areas in the image, capture global and local features, and improve the accuracy of the model.

[0064] Autoencoder model: It is an unsupervised learning algorithm whose main goal is to make the output value as similar as possible to the input value. The autoencoder model usually consists of two parts: an encoder and a decoder. The encoder is responsible for compressing the input data into a low-dimensional space, while the decoder attempts to reconstruct the original input from the compressed data. For image data, the encoder converts the high-dimensional pixel information of the image into a low-dimensional feature vector, which contains the key information of the image. The main task of the decoder is to reconstruct the original image based on the feature representation extracted by the encoder. The output of the decoder should be as close to the original input image as possible to ensure the effectiveness of the autoencoder model.

[0065] Secondly, describe the technical problems existing in related technologies:

[0066] With the rapid development of the Internet and the popularity of smart devices, the use and transmission of video data has become an indispensable part of our daily life. However, with it comes a large amount of video data that needs to be managed and processed. In order to improve the quality and security of video data, video anomaly detection technology has emerged.

[0067] In some application scenarios, it is necessary to identify faces through videos to achieve permission verification. In related technologies, permission verification is often performed by simple facial feature detection and then matching based on facial features. Assuming that the video used for facial feature detection is generated by obtaining a real face and then using a computer device to virtually generate it, the detection mechanism in the related technology cannot detect the authenticity of the video itself, and only considers the detection of facial features in the video. In this case, false videos may bypass the facial detection mechanism in the related technology, resulting in unreasonable authorization and ultimately causing property losses to users.

[0068] Therefore, the related technology cannot detect the authenticity of the video itself, and can only detect the content in the video.

[0069] For example, in some financial scenarios, when it comes to asset exchange, facial video verification is required, and asset exchange can only be achieved after verification. However, if the facial video is forged, the relevant technology cannot detect whether the facial video itself is a forged video. This may lead to incorrect authorization and illegal exchange of assets, thereby causing property loss to the user.

[0070] In order to solve the above technical problems, the present application proposes a solution for detecting the video itself through a trained video detection model, so as to determine whether there is any abnormality in the video, such as whether the video is a forged video.

[0071] In an embodiment of the present application, by obtaining a frame sequence to be detected of a video to be detected; inputting the frame sequence to be detected into a trained video detection model, outputting a target prediction frame sequence, the trained video detection model is obtained based on the attention features between the video frames of the training video; obtaining the similarity between the target prediction frame sequence and the frame sequence to be detected; comparing the similarity with a preset similarity threshold, when the similarity is greater than the preset similarity threshold, determining that the video to be detected is an abnormal video, and when the similarity is not greater than the preset similarity threshold, determining that the video to be detected is a normal video. In this way, through the attention mechanism in the trained video detection model, the feature extraction of the frame sequence to be detected of the video to be detected can be accurately performed, and then the extracted features are used to predict the target prediction frame sequence corresponding to the frame sequence to be detected, and finally by comparing the similarity between the frame sequence to be detected and the target prediction frame sequence, and comparing the similarity with the preset similarity threshold, it is possible to accurately detect whether the video to be detected is an abnormal video.

[0072] See also Figure 1 , Figure 1 It is a schematic diagram of a scene of video detection provided in an embodiment of the present application.

[0073] like Figure 1 As shown, assuming that the video detection scenario is a financial scenario of asset exchange, before the asset exchange, it is necessary to perform video detection on the recorded face video to determine whether the video is an abnormal video.

[0074] First, a video to be detected may be obtained, and then the video to be detected may be input into a trained video detection model. The trained video detection model includes a trained encoding sub-model, a trained attention sub-model, and a trained decoding sub-model.

[0075] The video to be detected is input into the trained encoding sub-model. The trained encoding model can extract features from each frame of the video to be detected, thereby obtaining the target encoding features of each video frame. Each target encoding feature is then input into the trained attention sub-model. The trained attention sub-model can extract the attention relationship between the target encoding features of each video frame, thereby obtaining the target attention features of each target encoding feature. Finally, the target attention features are input into the trained decoding sub-model. The trained decoding sub-model can predict the target prediction frame through each target attention feature, thereby obtaining a target prediction frame sequence.

[0076] It should be noted that the model type of the trained video detection model can be a self-encoding model, and the actual output of the trained video detection model is close to the input. Therefore, the target video frame sequence and the to-be-detected frame sequence of the to-be-detected video are similar.

[0077] It is understandable that the trained video detection model will have a certain model loss value, that is, when a normal video to be detected is input, the output and input of the trained video detection model must have a loss value greater than or equal to the model loss value. Therefore, a preset similarity threshold can be determined based on the model loss value. Assuming that the model loss value is loss, the preset similarity threshold is 1-loss.

[0078] The similarity between the target prediction frame sequence and the frame sequence to be detected can be obtained. When the similarity is not greater than a preset similarity threshold, the video to be detected is determined to be a normal video. When the similarity is greater than the preset similarity threshold, the video to be detected is determined to be an abnormal video.

[0079] That is to say, in the case of normal video input, the maximum similarity between the input and output is only the preset similarity threshold, and when the similarity is greater than the preset similarity threshold, it means that the video to be detected input into the trained video detection model is not normal, and may be an abnormal video forged by computer equipment. In this way, the trained video detection model can achieve accurate detection of the video to be detected, thereby ensuring the security of financial scenarios in asset exchange.

[0080] The embodiments of the present application provide a video detection method, apparatus, device and storage medium, which will be described in detail below.

[0081] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0082] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0083] The video detection method provided in the embodiment of the present application relates to the field of artificial intelligence technology. The video detection method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the video detection method, etc., but is not limited to the above forms.

[0084] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer computer devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0085] It should be noted that in each specific implementation of the present application, when it comes to relevant processing of training videos and video data to be detected, the user's permission or consent will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. In addition, when the embodiment of the present application needs to obtain sensitive personal information of the user, the user's separate permission or consent will be obtained through a pop-up window or jump to a confirmation page. After the user's separate permission or consent is clearly obtained, the necessary user-related data for the normal operation of the embodiment of the present application is obtained.

[0086] See also Figure 2 , Figure 2 : is a flow chart of the video detection method provided in the embodiment of the present application. The video detection method provided in the embodiment of the present application may include the following steps:

[0087] Step 110: Obtain a frame sequence to be detected of the video to be detected;

[0088] Step 120: input the frame sequence to be detected into the trained video detection model, and output the target prediction frame sequence, wherein the trained video detection model is trained based on the attention features between video frames of the training video;

[0089] Step 130, obtaining the similarity between the target prediction frame sequence and the frame sequence to be detected;

[0090] Step 140: Compare the similarity with a preset similarity threshold, and when the similarity is greater than the preset similarity threshold, determine that the video to be detected is an abnormal video;

[0091] Step 150: When the similarity is not greater than a preset similarity threshold, it is determined that the video to be detected is a normal video.

[0092] Steps 110 to 150 will be described in detail below.

[0093] In step 110, a frame sequence to be detected of a video to be detected is obtained.

[0094] The video to be detected can be a video input by the user in the client. For example, in a financial scenario, when a user needs to exchange assets, he needs to input a face video in the client for identity verification. Then, the frame sequence to be detected of the video to be detected is obtained, and the frame sequence to be detected can be arranged in chronological order.

[0095] In step 120, the frame sequence to be detected is input into a trained video detection model, and a target prediction frame sequence is output. The trained video detection model is trained based on attention features between video frames of a training video.

[0096] Among them, the trained video detection model can specifically be an autoencoding model. After inputting the frame sequence to be detected, the trained video detection model can output a target prediction frame sequence similar to the frame sequence to be detected.

[0097] The trained video detection model includes a trained encoding sub-model, a trained attention sub-model, and a trained decoding sub-model.

[0098] The video to be detected is input into the trained encoding sub-model. The trained encoding model can extract features from each frame of the video to be detected, thereby obtaining the target encoding features of each video frame. Each target encoding feature is then input into the trained attention sub-model. The trained attention sub-model can extract the attention relationship between the target encoding features of each video frame, thereby obtaining the target attention features of each target encoding feature. Finally, the target attention features are input into the trained decoding sub-model. The trained decoding sub-model can predict the target prediction frame through each target attention feature, thereby obtaining a target prediction frame sequence.

[0099] In order to understand the trained video detection model provided by the embodiment of the present application in more detail, the following first introduces the training method of the video detection model. The video detection model includes an encoding sub-model, an attention sub-model and a decoding sub-model.

[0100] Please also read Figure 3 , Figure 3 : is a schematic diagram of the training process of the video detection model provided in the embodiment of the present application. In some embodiments, before inputting the frame sequence to be detected into the trained video detection model and outputting the target prediction frame sequence, it also includes:

[0101] Step 201, obtaining a video frame sequence of a training video;

[0102] Step 202: input the video frame sequence into the coding sub-model, and output the coding features corresponding to each video frame in the video frame sequence;

[0103] Step 203: Input the encoded features into the attention sub-model to obtain the attention features between each video frame and other video frames;

[0104] Step 204: input the attention feature into the decoding sub-model and output the predicted video frame sequence;

[0105] Step 205, determine the loss value between the predicted video frame sequence and the video frame sequence. When the loss value is greater than the preset loss value, adjust the model parameters of the video detection model according to the loss value to implement iteration of the video detection model, and return to execute to obtain the video frame sequence of the training video until the loss value is no greater than the preset loss value, thereby obtaining the trained video detection model.

[0106] Steps 201 to 205 will be described in detail below.

[0107] In step 201, a video frame sequence of a training video is obtained.

[0108] The training video may be determined according to the actual usage scenario of the video detection model. For example, in a financial scenario, the training video may be a face video, which is a video authorized by the user. For example, in a media scenario, the training video may be a movie video.

[0109] After the training video is acquired, a video frame sequence of the training video may be acquired, and the video frame sequence may be arranged in chronological order.

[0110] Please also read Figure 4 , Figure 4 It is a structural diagram of the video detection model provided in the embodiment of the present application.

[0111] Among them, the video detection model is mainly divided into three parts, namely the encoding sub-model, the attention sub-model and the decoding sub-model. The video detection model can be understood as a self-encoding model, while the encoding sub-model and the attention sub-model are equivalent to the encoder, and the decoding sub-model is equivalent to the decoder.

[0112] The input of the encoding sub-model is the video frame sequence of the training video, and the encoding sub-model outputs the encoding features of each video frame. The input of the attention sub-model is each encoding feature, and the attention sub-model outputs the attention features of each video frame. The input of the decoding sub-model is the attention features of each video frame, and the decoding sub-model outputs the predicted video frame for each video frame, thereby obtaining a predicted video frame sequence.

[0113] In step 202, the video frame sequence is input into the encoding sub-model, and the encoding features corresponding to each video frame in the video frame sequence are output.

[0114] After obtaining the video frame sequence, the video frame sequence can be input into a coding sub-model. Specifically, the coding sub-model can be an encoder in a self-coding model. The coding sub-model can extract key information from each video frame in the video frame sequence, thereby obtaining coding features for each video frame in the video frame sequence.

[0115] In some implementations, considering that each video frame is actually an image, the encoding sub-model can be set from the perspective of image processing. The encoding sub-model can be constructed based on a convolutional neural network. Figure 5 , Figure 5 It is a structural diagram of the encoding sub-model provided in an embodiment of the present application.

[0116] The encoding sub-model includes multiple convolutional layers, multiple pooling layers and a fully connected layer. The output of the previous convolutional layer is connected to the input of the next pooling layer, the output of the next pooling layer is connected to the input of the next convolutional layer, and the last pooling layer is connected to the fully connected layer. For example, the input of the first convolutional layer is a video frame sequence, the output of the first convolutional layer is connected to the input of the first pooling layer, the output of the first pooling layer is connected to the input of the second convolutional layer, and so on. The output of the last pooling layer is connected to the fully connected layer.

[0117] Each video frame is an image. Through a series of convolutional layers and pooling layers, the spatial dimension of the input image is gradually reduced, the feature representation of the image is extracted, and the image is compressed into a low-dimensional latent vector. In other words, the key features of each video frame can be extracted through the encoding sub-model to obtain the encoding features of each video frame. The encoding features can be understood as the feature representation of each video frame.

[0118] In some implementations, a video frame sequence is input into a coding sub-model, and a coding feature corresponding to each video frame in the video frame sequence is output, including:

[0119] (1.1) Input the video frame sequence from the first convolutional layer of the encoding sub-model, and obtain the video frame sequence features output by the fully connected layer of the encoding sub-model;

[0120] (1.2) Perform dimensionality reduction processing on the features of the video frame sequence and output the encoding features corresponding to each video frame in the video frame sequence.

[0121] Among them, the video frame sequence is input from the first convolutional layer of the encoding sub-model, and then the features of the first convolutional layer are output, and the features are input into the first pooling layer. The subsequent convolutional layers and pooling layers process the intermediate features in turn until the output video frame sequence features of the fully connected layer of the encoding sub-model.

[0122] Among them, for each convolution layer, the convolution layer can automatically extract various features from the input image by using multiple different convolution kernels to perform convolution operations with the input data. For example, different convolution kernels can detect different image features such as edges, textures, and colors. The convolution kernel slides on the image and performs weighted summation on the local area to capture the local pattern and structural information of the image. This local connection method enables the convolution layer to effectively process data with spatial structure such as images.

[0123] For each pooling layer, the pooling layer downsamples the input data to reduce the spatial dimension of the data. For example, the maximum pooling layer selects the maximum value in the input area as the output, and the average pooling layer calculates the average value of the input area as the output. Dimensionality reduction can reduce the number of parameters and calculations of the model and improve the computational efficiency of the model. At the same time, dimensionality reduction can also help the model better capture the global characteristics of the data and avoid overfitting.

[0124] Finally, the fully connected layer can integrate the distributed feature representations learned by the previous layers into a global feature vector, thereby obtaining the video frame sequence features corresponding to the video frame sequence.

[0125] After obtaining the video frame sequence, the video frame sequence features can also be reduced in dimension to output the encoding features corresponding to each video frame in the video frame sequence. The advantage of this is that the video frame sequence features can be reduced in dimension again, thereby helping the model better capture the global features of the data and avoid overfitting.

[0126] In step 203, the encoded features are input into the attention sub-model to obtain the attention features between each video frame and other video frames.

[0127] It can be understood that after obtaining the coding features, we can further extract the attention relationship between the coding features of each video frame, so as to achieve feature enhanced representation of each video frame, that is, the attention features between each video frame relative to other video frames.

[0128] See also Figure 6 , Figure 6 : is a schematic diagram of the structure of the attention sub-model provided in the embodiment of the present application. Among them, the attention sub-model includes a plurality of attention modules connected in sequence. Each attention module can be a self-attention module, which has a self-attention mechanism. The long-range dependency between different frame positions in the training video can be captured through the self-attention mechanism.

[0129] In some embodiments, the encoded features are input into the attention sub-model to obtain the attention features of each video frame relative to other video frames, including:

[0130] (1.1) Input each encoded feature into the first attention module to obtain the sub-attention feature output by the first attention module;

[0131] (1.2) Input the sub-attention feature output by the first attention module into the next attention module to obtain the sub-attention feature output by the next attention module;

[0132] (1.3) Repeatedly input the sub-attention features output by the previous attention module into the next attention module until the last attention module outputs the attention features corresponding to each encoding feature, where the attention features are the attention features between each video frame relative to other video frames.

[0133] Among them, in the attention sub-model, the model structure of each attention module is the same, and will be described in detail from the perspective of the first attention module.

[0134] Each encoded feature can be input into the first attention module. The attention module has a self-attention network. The self-attention network has three channels, namely, Q channel, K channel and V channel, and different features can be input. Taking the encoded features of a certain video frame as an example, the encoded features of the video frame can be input into the Q channel, and the encoded features of other video frames can be input into the K channel and the V channel.

[0135] The Q channel outputs a query vector (Query), the K channel outputs a key vector (Key), and the V channel outputs a value vector (Value).

[0136] The query vector and key vector are used to calculate the similarity score between a certain coded feature and each coded feature among all coded features, and the value vector is the feature representation corresponding to each coded feature. The value vector and the above similarity score are used to calculate the weighted sum corresponding to a certain coded feature. The weighted sum can be understood as the attention vector corresponding to the coded feature. The coded feature and its related query vector, key vector and value vector are obtained by multiplying the coded feature by different weights.

[0137] For example, based on the query vector of a certain coding feature, the similarity between the query vector and the key vector of each coding feature is calculated to obtain the similarity score between the coding feature and each coding feature, and then the similarity score is multiplied by the value vector corresponding to each coding feature to obtain the multiplication result, and all the multiplication results are added together to obtain the attention vector corresponding to the coding feature.

[0138] The attention vector needs to be processed by the residual and normalization network and the feedforward neural network to obtain the sub-attention feature output by the first attention module.

[0139] Similarly, for other attention modules, the input of each attention module is the output of the previous attention module, and its specific data processing method is the same as that of the first attention module, which will not be repeated here.

[0140] Repeatedly input the sub-attention features output by the previous attention module into the next attention module until the last attention module outputs the attention features corresponding to each encoded feature, where the attention features are the attention features between each video frame and other video frames.

[0141] It can be understood that the attention feature of each video frame relative to other video frames is an enhanced representation of each coding feature. The coding feature is only obtained through the image information in each video frame, while the attention feature combines some feature information of other video frames. It contains the dependency relationship between other video frames and can more accurately represent the features of the video frame in the video frame sequence of the entire training video. Therefore, the attention feature of each video frame can more accurately represent the features of each video frame compared to the coding feature.

[0142] In step 204, the attention features are input into the decoding sub-model and a predicted video frame sequence is output.

[0143] Among them, the decoding sub-model can decode the attention features. The decoding sub-model gradually increases the dimension of the image through a series of deconvolution or upsampling operations, and restores the low-dimensional feature vector to high-dimensional image data. The output of the decoding sub-model should be as close to the original input image as possible to ensure the effectiveness of the autoencoder model.

[0144] During image reconstruction, the decoding sub-model needs to restore the detailed information in the image, including the color, texture, edge, etc. By learning the mapping relationship from feature vectors to images, the decoding sub-model can restore the details of the original image as much as possible, thereby improving the quality of the reconstructed image.

[0145] In the present application, by decoding the sub-model, it is possible to predict the predicted video frame of each video frame in the original training video.

[0146] Please also read Figure 7 , Figure 7 It is a schematic diagram of the structure of the decoding sub-model provided in an embodiment of the present application.

[0147] In some embodiments, the decoding sub-model includes a plurality of sequentially connected deconvolution layers and a fully connected layer, and the last deconvolution layer of the plurality of sequentially connected deconvolution layers is connected to the fully connected layer. Figure 7As shown in the figure, the input of the first deconvolution layer is the attention feature of each video frame, the input of other deconvolution layers is the output of the previous deconvolution layer, and the output of the last deconvolution layer is connected to the fully connected layer, which outputs the predicted video frame corresponding to each video frame in the training video.

[0148] In some embodiments, the attention feature is input into a decoding sub-model, and a predicted video frame sequence is output, including:

[0149] (1.1) Input the attention features into multiple sequentially connected deconvolution layers to obtain the decoding features corresponding to each video frame;

[0150] (1.2) Input the decoded features into the fully connected layer and output the predicted video frame sequence.

[0151] Among them, when the decoding sub-model processes the video frame sequence of the training video, it will undergo a series of convolution operations to gradually reduce the size of the video frame. The deconvolution layer can reverse this process and increase the resolution by increasing the size of the video frame. The decoding sub-model needs to generate a high-resolution output image from the low-resolution attention features, that is, the predicted video frame. The deconvolution layer can gradually enlarge the image and fill in more details, making the output image clearer.

[0152] In addition, some details may be lost during the image compression or downsampling process of the decoding sub-model. The deconvolution layer can restore these lost details by learning the mapping relationship from low resolution to high resolution. The deconvolution layer can reconstruct the texture, edges and other details in the image, making the image closer to the original high-resolution version, that is, making the predicted video frame closer to the video frame of the training video.

[0153] The last deconvolution layer outputs the decoded features corresponding to each video frame, and then inputs the decoded features into the fully connected layer to output a predicted video frame sequence. The predicted video frame sequence can be understood as a video frame sequence that is close to the video frame sequence of the training video.

[0154] In step 205, the loss value between the predicted video frame sequence and the video frame sequence is determined. When the loss value is greater than the preset loss value, the model parameters of the video detection model are adjusted according to the loss value to implement iteration of the video detection model, and the execution is returned to obtain the video frame sequence of the training video until the loss value is no greater than the preset loss value, thereby obtaining the trained video detection model.

[0155] It is determined that the predicted video frame sequence and the video frame sequence are not completely the same, so a loss value between the predicted video frame sequence and the video frame sequence can be determined.

[0156] When the loss value is greater than the preset loss value, it is considered that the video detection model has not converged. At this time, the video detection model needs to be trained, that is, the encoding sub-model, attention sub-model and decoding sub-model in the video detection model need to continue to be trained. At this time, the model parameters of the video detection model are adjusted according to the loss value to achieve iteration of the video detection model, such as adjusting some model parameters of the encoding sub-model, attention sub-model and decoding sub-model, so as to achieve iteration of the video detection model.

[0157] Then return to obtain the video frame sequence of the training video, that is, return to execute step 201, and continue to start the next training process until the loss value between the predicted video frame sequence output by the video detection model and the video frame sequence is less than or equal to the preset loss value, thereby obtaining a converged video detection model, that is, a trained video detection model.

[0158] From step 201 to step 205, it can be seen that in the present application, in the stage of training the video detection model, the attention sub-model is used to enhance the encoding features output by the encoding sub-model to obtain the attention features. In this way, the features of each video frame in the training video frame can be more accurately represented, so that the decoding sub-model can more accurately obtain the predicted video frame sequence according to the attention features. The predicted video frame sequence is closer to the video frame sequence of the training video.

[0159] It should be noted that the training videos are real videos, so the trained video detection model can generate predicted videos that are closest to the real videos.

[0160] Therefore, the frame sequence to be detected is input into the trained video detection model, and the target prediction frame sequence is output. The output target prediction frame sequence can be considered to be the closest to the frame sequence to be detected. However, since the trained video detection model still has a certain model loss value, there will be a corresponding maximum preset similarity threshold between the target prediction frame sequence and the frame sequence to be detected. Assuming the model loss value is loss, the preset similarity threshold is 1-loss.

[0161] In step 130, the similarity between the target prediction frame sequence and the frame sequence to be detected is obtained.

[0162] The frame images at the same time in the target prediction frame sequence and the frame sequence to be detected may be compared to obtain the similarity between the target prediction frame sequence and the frame sequence to be detected.

[0163] In some implementations, obtaining the similarity between the target prediction frame sequence and the to-be-detected frame sequence includes:

[0164] (1.1) determining a target prediction frame in the target prediction frame sequence, and determining a target video frame corresponding to each target prediction frame in the frame sequence to be detected;

[0165] (1.2) determining the frame similarity between each target prediction frame and the target video frame corresponding to each target prediction frame;

[0166] (1.3) The similarity between the target prediction frame sequence and the frame sequence to be detected is determined based on the frame similarity.

[0167] Among them, some frames are extracted from the target prediction frame sequence as target prediction frames, and the time point corresponding to each target prediction frame is determined, and then the target video frame corresponding to each target prediction frame is determined in the frame sequence to be detected according to the time point.

[0168] Then determine the frame similarity between each target prediction frame and the target video frame corresponding to each target prediction frame. For example, between the target prediction frame and the target video frame at time point 1, image similarity analysis can be performed to obtain the similarity between the two.

[0169] Finally, the similarity between the target prediction frame sequence and the frame sequence to be detected is determined based on the frame similarity between each target prediction frame and the target video frame. In this way, the similarity between the target prediction frame sequence and the frame sequence to be detected can be quickly determined, thereby improving the efficiency of video detection.

[0170] In some implementations, determining the similarity between the target prediction frame sequence and the frame sequence to be detected according to the frame similarity includes:

[0171] (1.3.1) Determine the number of frames in the target prediction frame sequence;

[0172] (1.3.2) Add the frame similarities corresponding to each target prediction frame and divide by the number of frames to obtain the similarity between the target prediction frame sequence and the frame sequence to be detected.

[0173] For example, it is determined that the number of frames in the target prediction frame sequence is 10, and then the frame similarities corresponding to each target prediction frame are added to obtain an addition result, and finally the addition result is divided by the number of frames 10, then the similarity between the target prediction frame sequence and the frame sequence to be detected is obtained. The advantage of this is that the similarity between the target prediction frame sequence and the frame sequence to be detected can be measured by the frame similarities corresponding to multiple target prediction frames. The accuracy of determining the similarity between the target prediction frame sequence and the frame sequence to be detected can be improved.

[0174] In step 140, the similarity is compared with a preset similarity threshold, and when the similarity is greater than the preset similarity threshold, it is determined that the video to be detected is an abnormal video.

[0175] It is understandable that the training videos are real videos, so the trained video detection model can generate a predicted video that is closest to the real video.

[0176] If the video to be detected is a real video, then the similarity between the target predicted frame sequence and the frame sequence to be detected must be or equal to the preset similarity threshold, because the preset similarity threshold is the highest similarity performance between the output result of the trained video detection model and the input.

[0177] Therefore, when the similarity is greater than a preset similarity threshold, the video to be detected is determined to be an abnormal video, indicating that the video to be detected is not a real shot video, but may be a video generated by a computer device.

[0178] In step 150, when the similarity is not greater than a preset similarity threshold, it is determined that the video to be detected is a normal video.

[0179] When the similarity is not greater than the preset similarity threshold, it indicates that the video to be detected is a real shot video, and the video to be detected is determined to be a normal video.

[0180] Therefore, in the application, the trained video detection model can be used to accurately determine whether the video to be detected is a normal video or an abnormal video, which can improve the accuracy of video detection.

[0181] As can be seen from the above, in the embodiment of the present application, by obtaining the frame sequence to be detected of the video to be detected; inputting the frame sequence to be detected into the trained video detection model, outputting the target prediction frame sequence, the trained video detection model is obtained based on the attention feature training between the video frames of the training video; obtaining the similarity between the target prediction frame sequence and the frame sequence to be detected; comparing the similarity with the preset similarity threshold, when the similarity is greater than the preset similarity threshold, determining that the video to be detected is an abnormal video, when the similarity is not greater than the preset similarity threshold, determining that the video to be detected is a normal video. In this way, through the trained video detection model, the feature extraction of the frame sequence to be detected of the video to be detected can be accurately performed through the attention mechanism therein, and then the extracted features are used to predict the target prediction frame sequence corresponding to the frame sequence to be detected, and finally by comparing the similarity between the frame sequence to be detected and the target prediction frame sequence, and comparing the similarity with the preset similarity threshold, it is possible to accurately detect whether the video to be detected is an abnormal video.

[0182] See also Figure 8 , Figure 8 : is another flow chart of the video detection method provided in the embodiment of the present application. The video detection method may include the following steps:

[0183] Step 301, obtaining a video frame sequence of a training video;

[0184] Step 302: Input the video frame sequence into the coding sub-model, and output the coding features corresponding to each video frame in the video frame sequence;

[0185] Step 303: Input the encoded features into the attention sub-model to obtain the attention features of each video frame relative to other video frames;

[0186] Step 304: input the attention features into the decoding sub-model and output a predicted video frame sequence;

[0187] Step 305: determine the loss value between the predicted video frame sequence and the video frame sequence. When the loss value is greater than the preset loss value, adjust the model parameters of the video detection model according to the loss value to implement iteration of the video detection model, and return to execute to obtain the video frame sequence of the training video until the loss value is no greater than the preset loss value, thereby obtaining the trained video detection model.

[0188] Step 306: Obtain a frame sequence to be detected of the video to be detected;

[0189] Step 307: input the frame sequence to be detected into the trained video detection model, and output the target prediction frame sequence;

[0190] Step 308: determine a target prediction frame in the target prediction frame sequence, and determine a target video frame corresponding to each target prediction frame in the to-be-detected frame sequence;

[0191] Step 309: determining the frame similarity between each target prediction frame and the target video frame corresponding to each target prediction frame;

[0192] Step 310: determine the number of frames of the target prediction frame sequence, add the frame similarities corresponding to each target prediction frame and divide the sum by the number of frames to obtain the similarity between the target prediction frame sequence and the frame sequence to be detected;

[0193] Step 311: compare the similarity with a preset similarity threshold. When the similarity is greater than the preset similarity threshold, determine that the video to be detected is an abnormal video. When the similarity is not greater than the preset similarity threshold, determine that the video to be detected is a normal video.

[0194] In the above embodiments, the description of each embodiment has its own emphasis. For the part that is not described in detail in a certain embodiment, please refer to the detailed description of the above video detection method, which will not be repeated here.

[0195] See also Fig. 9 , Fig. 9 It is a structural schematic diagram of a video detection device provided in an embodiment of the present application.

[0196] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0197] like Fig. 9 As shown, the video detection device 400 includes:

[0198] A first acquisition module 410 is used to acquire a frame sequence to be detected of a video to be detected;

[0199] An input module 420 is used to input a frame sequence to be detected into a trained video detection model and output a target prediction frame sequence, wherein the trained video detection model is trained based on attention features between video frames of a training video;

[0200] A second acquisition module 430 is used to acquire the similarity between the target prediction frame sequence and the frame sequence to be detected;

[0201] A first determination module 440 is used to compare the similarity with a preset similarity threshold, and when the similarity is greater than the preset similarity threshold, determine that the video to be detected is an abnormal video;

[0202] The second determination module 450 is configured to determine that the video to be detected is a normal video when the similarity is not greater than a preset similarity threshold.

[0203] In some embodiments, the second acquisition module 430 includes a first determination submodule 431 , a second determination submodule 432 , and a third determination submodule 433 ;

[0204] A first determination submodule 431 is used to determine a target prediction frame in a target prediction frame sequence, and to determine a target video frame corresponding to each target prediction frame in a frame sequence to be detected;

[0205] A second determination submodule 432 is used to determine the frame similarity between each target prediction frame and the target video frame corresponding to each target prediction frame;

[0206] The third determination submodule 433 is used to determine the similarity between the target prediction frame sequence and the to-be-detected frame sequence according to the frame similarity.

[0207] In some implementations, the third determination submodule 433 is used to determine the number of frames of the target prediction frame sequence;

[0208] The frame similarities corresponding to each target prediction frame are added and divided by the number of frames to obtain the similarity between the target prediction frame sequence and the frame sequence to be detected.

[0209] In some implementations, the video detection apparatus 400 further includes a training module 460 for:

[0210] Get a video frame sequence of a training video;

[0211] Input the video frame sequence into the encoding sub-model, and output the encoding features corresponding to each video frame in the video frame sequence;

[0212] Input the encoded features into the attention sub-model to obtain the attention features between each video frame and other video frames;

[0213] Input the attention features into the decoding sub-model and output the predicted video frame sequence;

[0214] Determine the loss value between the predicted video frame sequence and the video frame sequence. When the loss value is greater than the preset loss value, adjust the model parameters of the video detection model according to the loss value to implement the iteration of the video detection model, and return to execute to obtain the video frame sequence of the training video until the loss value is no greater than the preset loss value, and obtain the trained video detection model.

[0215] In some embodiments, the encoding sub-model includes multiple convolutional layers, multiple pooling layers and a fully connected layer, the output of the previous convolutional layer is connected to the input of the next pooling layer, the output of the next pooling layer is connected to the input of the next convolutional layer, and the last pooling layer is connected to the fully connected layer; the training module 460 is used to:

[0216] Input the video frame sequence from the first convolutional layer of the encoding sub-model, and obtain the video frame sequence features output by the fully connected layer of the encoding sub-model;

[0217] Perform dimensionality reduction processing on the video frame sequence features and output the encoding features corresponding to each video frame in the video frame sequence.

[0218] In some embodiments, the attention sub-model includes multiple attention modules, a training module 460, for:

[0219] Input each encoded feature into the first attention module to obtain the sub-attention feature output by the first attention module;

[0220] Input the sub-attention features output by the first attention module into the next attention module to obtain the sub-attention features output by the next attention module;

[0221] Repeatedly input the sub-attention features output by the previous attention module into the next attention module until the last attention module outputs the attention features corresponding to each encoded feature, where the attention features are the attention features between each video frame and other video frames.

[0222] In some embodiments, the decoding sub-model includes a plurality of sequentially connected deconvolution layers and a fully connected layer, and the last deconvolution layer of the plurality of sequentially connected deconvolution layers is connected to the fully connected layer; the training module 460 is used to:

[0223] The attention features are input into the decoding sub-model, and the output is a predicted video frame sequence, including:

[0224] The attention features are input into multiple sequentially connected deconvolution layers to obtain the decoding features corresponding to each video frame;

[0225] The decoded features are input into the fully connected layer and the predicted video frame sequence is output.

[0226] In the above embodiments, the description of each embodiment has its own emphasis. For the part that is not described in detail in a certain embodiment, please refer to the detailed description of the above video detection method, which will not be repeated here.

[0227] As can be seen from the above, in the embodiment of the present application, the first acquisition module 410 acquires the frame sequence to be detected of the video to be detected; the input module 420 inputs the frame sequence to be detected into the trained video detection model, and outputs the target prediction frame sequence, and the trained video detection model is obtained based on the attention feature training between the video frames of the training video; the second acquisition module 430 acquires the similarity between the target prediction frame sequence and the frame sequence to be detected; the first determination module 440 compares the similarity with the preset similarity threshold, and when the similarity is greater than the preset similarity threshold, the video to be detected is determined to be an abnormal video; the second determination module 450 determines that the video to be detected is a normal video when the similarity is not greater than the preset similarity threshold. In this way, through the trained video detection model, the feature extraction of the frame sequence to be detected of the video to be detected can be accurately performed through the attention mechanism therein, and then the extracted features are used to predict the target prediction frame sequence corresponding to the frame sequence to be detected, and finally by comparing the similarity between the frame sequence to be detected and the target prediction frame sequence, and comparing the similarity with the preset similarity threshold, it is possible to accurately detect whether the video to be detected is an abnormal video.

[0228] See also Fig.10 The embodiment of the present application also provides a computer device, the computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the above video detection method when executing the computer program. The computer device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0229] See also Fig.10 , Fig.10 The hardware structure of a computer device according to another embodiment is shown, and the computer device includes:

[0230] The processor 501 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0231] The memory 502 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 502 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 502, and the processor 501 is called to execute the video detection method of the embodiment of this application;

[0232] Input / output interface 503, used to implement information input and output;

[0233] Communication interface 504, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0234] A bus 505 that transmits information between the various components of the device (e.g., the processor 501, the memory 502, the input / output interface 503, and the communication interface 504);

[0235] The processor 501 , the memory 502 , the input / output interface 503 and the communication interface 504 are connected to each other in communication within the device via the bus 505 .

[0236] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned video detection method is implemented.

[0237] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0238] The video detection method, device, equipment and storage medium provided in the embodiment of the present application obtain the frame sequence to be detected of the video to be detected; input the frame sequence to be detected into the trained video detection model, output the target prediction frame sequence, the trained video detection model is trained based on the attention features between the video frames of the training video; obtain the similarity between the target prediction frame sequence and the frame sequence to be detected; compare the similarity with the preset similarity threshold, when the similarity is greater than the preset similarity threshold, determine that the video to be detected is an abnormal video, when the similarity is not greater than the preset similarity threshold, determine that the video to be detected is a normal video. In this way, through the trained video detection model, the feature extraction of the frame sequence to be detected of the video to be detected can be accurately performed through the attention mechanism therein, and then the extracted features are used to predict the target prediction frame sequence corresponding to the frame sequence to be detected, and finally by comparing the similarity between the frame sequence to be detected and the target prediction frame sequence, and comparing the similarity with the preset similarity threshold, it is possible to accurately detect whether the video to be detected is an abnormal video.

[0239] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0240] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0241] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0242] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0243] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0244] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0245] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0246] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0247] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0248] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0249] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A video detection method, characterized in that: include: Obtain a frame sequence to be detected of a video to be detected; Inputting the frame sequence to be detected into a trained video detection model, and outputting a target prediction frame sequence, wherein the trained video detection model is trained based on attention features between video frames of a training video; Obtaining the similarity between the target prediction frame sequence and the frame sequence to be detected; Comparing the similarity with a preset similarity threshold, and when the similarity is greater than the preset similarity threshold, determining that the video to be detected is an abnormal video; When the similarity is not greater than the preset similarity threshold, it is determined that the video to be detected is a normal video.

2. The video detection method according to claim 1, characterized in that: The obtaining the similarity between the target prediction frame sequence and the to-be-detected frame sequence includes: Determining a target prediction frame in the target prediction frame sequence, and determining a target video frame corresponding to each target prediction frame in the to-be-detected frame sequence; Determining a frame similarity between each target prediction frame and a target video frame corresponding to each target prediction frame; The similarity between the target prediction frame sequence and the to-be-detected frame sequence is determined according to the frame similarity.

3. The video detection method according to claim 2, characterized in that: The determining, according to the frame similarity, the similarity between the target prediction frame sequence and the to-be-detected frame sequence comprises: Determining the number of frames of the target prediction frame sequence; The frame similarities corresponding to each target prediction frame are added and divided by the number of frames to obtain the similarity between the target prediction frame sequence and the frame sequence to be detected.

4. The video detection method according to claim 1, characterized in that: The video detection model includes an encoding sub-model, an attention sub-model and a decoding sub-model. Before inputting the frame sequence to be detected into the trained video detection model and outputting the target prediction frame sequence, it also includes: Get a video frame sequence of a training video; Inputting the video frame sequence into the coding sub-model, and outputting the coding features corresponding to each video frame in the video frame sequence; Inputting the encoded features into the attention sub-model to obtain attention features between each video frame and other video frames; Inputting the attention features into the decoding sub-model and outputting a predicted video frame sequence; Determine the loss value between the predicted video frame sequence and the video frame sequence. When the loss value is greater than a preset loss value, adjust the model parameters of the video detection model according to the loss value to implement iteration of the video detection model, and return to execute the video frame sequence of the training video until the loss value is no greater than the preset loss value, thereby obtaining a trained video detection model.

5. The video detection method according to claim 4, characterized in that: The encoding sub-model includes multiple convolutional layers, multiple pooling layers and a fully connected layer, the output of the previous convolutional layer is connected to the input of the next pooling layer, the output of the next pooling layer is connected to the input of the next convolutional layer, and the last pooling layer is connected to the fully connected layer; The step of inputting the video frame sequence into the coding sub-model and outputting the coding features corresponding to each video frame in the video frame sequence comprises: Input the video frame sequence from the first convolutional layer of the encoding sub-model, and obtain the video frame sequence features output by the fully connected layer of the encoding sub-model; Perform dimensionality reduction processing on the features of the video frame sequence, and output the encoding features corresponding to each video frame in the video frame sequence.

6. The video detection method according to claim 4, characterized in that: The attention sub-model includes a plurality of attention modules, and the encoding feature is input into the attention sub-model to obtain the attention feature between each video frame and other video frames, including: Input each of the encoded features into the first attention module to obtain a sub-attention feature output by the first attention module; Inputting the sub-attention feature output by the first attention module into the next attention module to obtain the sub-attention feature output by the next attention module; Repeatedly input the sub-attention features output by the previous attention module into the next attention module until the last attention module outputs the attention features corresponding to each of the encoded features, where the attention features are the attention features between each video frame relative to other video frames.

7. The video detection method according to claim 4, characterized in that: The decoding sub-model includes a plurality of sequentially connected deconvolution layers and a fully connected layer, and the last deconvolution layer of the plurality of sequentially connected deconvolution layers is connected to the fully connected layer; The step of inputting the attention feature into the decoding sub-model and outputting a predicted video frame sequence comprises: Inputting the attention features into a plurality of sequentially connected deconvolution layers to obtain decoding features corresponding to each video frame; The decoded features are input into the fully connected layer, and a predicted video frame sequence is output.

8. A video detection device, characterized in that: include: A first acquisition module is used to acquire a frame sequence to be detected of a video to be detected; An input module, used to input the frame sequence to be detected into a trained video detection model, and output a target prediction frame sequence, wherein the trained video detection model is trained based on attention features between video frames of a training video; A second acquisition module, used to acquire the similarity between the target prediction frame sequence and the to-be-detected frame sequence; A first determination module is used to compare the similarity with a preset similarity threshold, and when the similarity is greater than the preset similarity threshold, determine that the video to be detected is an abnormal video; The second determination module is used to determine that the video to be detected is a normal video when the similarity is not greater than the preset similarity threshold.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the video detection method according to any one of claims 1 to 7.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the video detection method according to any one of claims 1 to 7 is implemented.

Images