Video data processing method, device, computer equipment and storage medium

Abstract

The present application relates to a video data processing method, apparatus, computer equipment and storage medium. The method comprises: obtaining a video frame sequence containing a document to be identified; determining the position of the document to be identified in the video frame sequence; determining multiple position states of the document to be identified in the video frame sequence based on the position of the document to be identified in the video frame sequence; matching each of the position states with a target position state corresponding to a preset action, and determining the video frame corresponding to the successfully matched position state as a candidate key frame to be identified; the preset action is an action that causes the anti-counterfeiting point of a standard document to change, and the type of the standard document is consistent with that of the document to be identified; and determining the identification result of the document to be identified based on the candidate key frame. The use of this method can expand the application scope of document authenticity identification.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a video data processing method, apparatus, computer equipment, and storage medium. Background Art

[0002] Documents are commonly used to prove a user's identity, including but not limited to ID cards, passports, teacher IDs, employee IDs, etc. In various situations where documents are used, it is usually necessary to verify the authenticity of the documents to ensure information security.

[0003] Traditionally, when verifying the authenticity of an ID, the image in the ID's chip is read while a scanned image of the ID is obtained. The two are then compared to verify the ID's authenticity. However, this method relies on the physical ID and has a limited scope of application. Summary of the Invention

[0004] Based on this, it is necessary to provide a video data processing method, device, computer equipment and storage medium that can expand the application scope of document authenticity identification in order to address the above technical problems.

[0005] A video data processing method, characterized in that the method comprises:

[0006] Obtain a video frame sequence containing the document to be authenticated;

[0007] Determining the position of the document to be authenticated in the video frame sequence;

[0008] determining a plurality of position states of the certificate to be authenticated in the video frame sequence according to the position of the certificate to be authenticated in the video frame sequence;

[0009] Matching each of the position states with a target position state corresponding to a preset action, and determining the video frame corresponding to the successfully matched position state as a candidate key frame to be identified; the preset action is an action that causes the anti-counterfeiting point of the standard document to change, and the standard document is of the same type as the document to be identified;

[0010] An authentication result of the document to be authenticated is determined based on the candidate key frames.

[0011] In one embodiment, the first convolutional layer includes multiple convolutional layers with successively smaller scales; and inputting the template image and the search image into the first convolutional layer, and performing feature extraction on the template image and the search image respectively through the first convolutional layer includes:

[0012] The template image is subjected to multiple feature extractions through the first convolutional layer. Each time feature extraction is performed, the output features of the previous convolutional layer are fused with the input features to obtain the current input features. The current input features are input into the current convolutional layer, and convolution processing is performed through the current convolutional layer.

[0013] A video data processing device, characterized in that the device comprises:

[0014] A video frame acquisition module, used to acquire a video frame sequence containing the document to be authenticated;

[0015] A position determination module, configured to determine the position of the document to be authenticated in the video frame sequence;

[0016] a position state determining module, configured to determine a plurality of position states of the document to be authenticated in the video frame sequence according to the position of the document to be authenticated in the video frame sequence;

[0017] a matching module for matching each of the position states with a target position state corresponding to a preset action, and determining a video frame corresponding to a successfully matched position state as a candidate key frame to be identified; the preset action is an action that causes an anti-counterfeiting point of a standard document to change, and the standard document and the document to be identified are of the same type;

[0018] The identification result determination module is used to determine the identification result of the document to be identified based on the candidate key frames.

[0019] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0020] Obtain a video frame sequence containing the document to be authenticated;

[0021] Determining the position of the document to be authenticated in the video frame sequence;

[0022] determining a plurality of position states of the certificate to be authenticated in the video frame sequence according to the position of the certificate to be authenticated in the video frame sequence;

[0023] Matching each of the position states with a target position state corresponding to a preset action, and determining the video frame corresponding to the successfully matched position state as a candidate key frame to be identified; the preset action is an action that causes the anti-counterfeiting point of the standard document to change, and the standard document is of the same type as the document to be identified;

[0024] An authentication result of the document to be authenticated is determined based on the candidate key frames.

[0025] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0026] Obtain a video frame sequence containing the document to be authenticated;

[0027] Determining the position of the document to be authenticated in the video frame sequence;

[0028] determining a plurality of position states of the certificate to be authenticated in the video frame sequence according to the position of the certificate to be authenticated in the video frame sequence;

[0029] Matching each of the position states with a target position state corresponding to a preset action, and determining the video frame corresponding to the successfully matched position state as a candidate key frame to be identified; the preset action is an action that causes the anti-counterfeiting point of the standard document to change, and the standard document is of the same type as the document to be identified;

[0030] An authentication result of the document to be authenticated is determined based on the candidate key frames.

[0031] The above-mentioned video data processing method, apparatus, computer equipment and storage medium obtain a video frame sequence containing the certificate to be identified, and further determine the position of the certificate to be identified in the video frame sequence. According to the position of the certificate to be identified in the video frame sequence, multiple position states of the certificate to be identified in the video frame sequence are determined, and each position state is matched with a target position state corresponding to a preset action. The video frame corresponding to the successfully matched position state is determined as a candidate key frame to be identified. Since the preset action is an action that causes the anti-counterfeiting point of the standard certificate to change, and the type of the standard certificate is consistent with that of the certificate to be identified, the identification result of the certificate to be identified can be determined based on the candidate key frame. In the embodiment of the present application, when the authenticity of the certificate is identified, only the video frame sequence needs to be obtained, which is independent of the certificate entity, greatly expanding the application scope of the authenticity identification of the certificate. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A diagram showing an application environment of a video data processing method in one embodiment;

[0033] Figure 2 1 is a flow chart of a method for processing video data in one embodiment;

[0034] Figure 3 FIG1 is a schematic diagram of a process for determining an initial video frame in one embodiment;

[0035] Figure 4 A schematic diagram of a flow chart for determining the position of the document to be authenticated in a video frame sequence in another embodiment;

[0036] Figure 5 A schematic diagram of a target tracking process using a target tracking model in one embodiment;

[0037] Figure 6 A schematic diagram of a process for determining an identification result in one embodiment;

[0038] Figure 7 A schematic diagram of a process flow for obtaining a target key frame in one embodiment;

[0039] Figure 8 is a structural block diagram of a video data processing device in one embodiment;

[0040] Figure 9 is a diagram of the internal structure of a computer device in one embodiment;

[0041] Figure 10 Schematic diagram of the network structure of a target detection model in one embodiment. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0043] Artificial Intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also involves studying the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0044] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0045] Computer vision (CV) is the science of making machines "see." Specifically, it refers to the use of cameras and computers to replace the human eye in identifying, tracking, and measuring objects. This involves further processing the images, transforming them into images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0046] Machine learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0047] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, autonomous driving, drones, robots, smart medical care, smart customer service, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0048] The solutions provided in the embodiments of the present application involve technologies such as artificial intelligence computer vision, which are specifically described through the following embodiments:

[0049] The video data processing method provided in this application can be applied to Figure 1In the application environment shown. Both the terminal 102 and the server 104 can be used alone to execute the video data processing method provided in the embodiment of the present application. The terminal 102 and the server 104 can also be used in conjunction to execute the video data processing and generation method provided in the embodiment of the present application. For example, the terminal can obtain a video frame sequence containing the certificate to be identified through video shooting, and send the obtained video frame sequence to the server. The server determines the position of the certificate to be identified in the video frame sequence, and determines multiple position states of the certificate to be identified in the video frame sequence based on the position of the certificate to be identified in the video frame sequence. Each position state is matched with the target position state corresponding to the preset action, and the video frame corresponding to the successfully matched position state is determined as the candidate key frame to be identified. Based on the candidate key frame, the identification result of the certificate to be identified is determined, and finally the identification result is returned to the terminal 102. The preset action here is an action that causes the anti-counterfeiting point of the standard certificate to change, and the type of the standard certificate is consistent with that of the certificate to be identified.

[0050] The server 104 may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal may be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to these. The terminal and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in this application.

[0051] In one embodiment, Figure 2 As shown, a video data processing method is provided, which is applied to Figure 1 The following steps are used as an example to illustrate the terminal in the figure:

[0052] Step 202: Obtain a video frame sequence containing the document to be authenticated.

[0053] The document to be authenticated refers to a document that needs to be verified for authenticity. Depending on the business scenario, the document can be any document with anti-counterfeiting features. For example, the document to be authenticated can be any ID card, such as a second-generation ID card, a Hong Kong ID card, a Malaysian ID card, and the like.

[0054] Specifically, the video frame sequence can be a video captured by the terminal. For example, the terminal can prompt the user to capture a video while holding the ID to be authenticated. During the capture process, a prompt message is displayed, prompting the user to perform a preset action on the ID to be authenticated. The prompt message can include at least one of a voice prompt and a text prompt. The video frame sequence can also be a video captured by the terminal from another computer device. For example, a user can capture a video frame sequence using another computer device and transmit it to the terminal via a network or Bluetooth.

[0055] Step 204: Determine the position of the document to be authenticated in the video frame sequence.

[0056] The position of the document to be authenticated in the video frame sequence refers to the position coordinates of the document to be authenticated in the video frames included in the video frame sequence. In specific implementation, the position of the document to be authenticated in the video frame sequence is usually represented by the coordinates of four vertices of the image area corresponding to the document to be authenticated.

[0057] In one embodiment, not every frame in the video frame sequence captured by the terminal contains the ID to be authenticated. Therefore, when determining the location of the ID to be authenticated, the terminal may discard video frames that do not contain the ID to be authenticated and determine the location of the ID to be authenticated in the remaining video frames. For video frames containing the ID to be authenticated, each video frame will correspond to a location.

[0058] It is understood that the positions of the video frames containing the ID to be authenticated may be the same or different. Generally speaking, the positions of consecutive video frames within a short period of time do not change much and can be approximately considered the same. Therefore, in one embodiment, to improve processing efficiency, the terminal may determine the position every preset frame. The preset frame here can be set by relevant technicians based on experience.

[0059] In one embodiment, the terminal can perform line detection on the video frames in the video frame sequence, and combine the lines corresponding to the four edges of the ID to be authenticated obtained by the detection to obtain the position coordinates of the ID to be authenticated in the video frames. The terminal can use the Hough line detection algorithm when performing line detection.

[0060] In another embodiment, the terminal can determine the position coordinates of the document to be identified from the video frames in the video frame sequence using a trained machine learning model. For example, the terminal can use a trained machine learning model for image segmentation to perform image segmentation on the video frames in the video frame sequence to determine the position coordinates of the document to be identified from each video frame in the video frame sequence. For another example, the terminal can use a trained machine learning model for target detection to perform target detection on each video frame in the video frame sequence to determine the position coordinates of the document to be identified from the video frames in the video frame sequence. For another example, the terminal can use a trained machine learning model for target tracking to perform target tracking on the video frames in the video frame sequence to determine the position of the document to be identified in the video frame sequence.

[0061] Step 206: Determine multiple position states of the document to be authenticated in the video frame sequence according to the position of the document to be authenticated in the video frame sequence.

[0062] The position state refers to the state information corresponding to the position of the ID to be authenticated in the video frame. The position state corresponds to a preset action. Depending on the preset action, the position state can include a flipped state and a flat state. The flipped state describes the state of the ID to be authenticated after being flipped, while the flat state describes the state of the ID to be authenticated after being laid flat. The flipped state includes direction information and angle information. The direction information describes the flipping direction, while the angle information describes the flipping angle.

[0063] Specifically, the terminal can pre-acquire the mapping relationship between the position and the position state. After determining the position of the document to be identified in each frame of the video frame sequence, the terminal can determine the position state of the document to be identified in each frame of the video frame according to the mapping relationship, and obtain the position state corresponding to each frame of the video frame.

[0064] In one embodiment, the terminal may pre-train a machine learning model for predicting position status, and learn the mapping relationship between position and position status through the machine learning model. Then, when determining the position status, the terminal may input the position of the document to be identified in each frame of the video into the machine learning model respectively, and determine the position status of each frame of the video according to the output result of the machine learning model.

[0065] In step 208 , each position state is matched with a target position state corresponding to a preset action, and a video frame corresponding to a successfully matched position state is determined as a candidate key frame to be identified.

[0066] Among them, the preset action is an action that causes the anti-counterfeiting points of the standard certificate to change. The standard certificate here is a genuine certificate and is consistent with the type of the certificate to be identified. The anti-counterfeiting points of the certificate usually include at least one of dynamic anti-counterfeiting points and static anti-counterfeiting points. Dynamic anti-counterfeiting points are, for example, color-changing ink, dynamic printed fonts, dynamic printed portraits, etc. Dynamic anti-counterfeiting points can change when the user performs a flipping action, while static anti-counterfeiting points can change when the certificate is laid flat. Then, for certificates containing dynamic anti-counterfeiting points, depending on the changes in the dynamic anti-counterfeiting points of different certificates, the predicted action may include one or more of flipping up, flipping down, flipping to the left, and flipping to the right. For certificates containing static anti-counterfeiting points, the preset action includes laying flat. It can be understood that flipping and laying flat here are both ways of presenting the certificate in the video screen.

[0067] The target position state corresponding to a preset action refers to the desired position state of the document to be authenticated when the preset action is performed on the document. In this target position state, the security features of the standard document will change significantly. The target position state can be pre-set by relevant technical personnel based on experience. For example, if the preset action is a flip-up action, the target position state corresponding to this flip-up action may include the desired angle information of the document to be authenticated when the preset action is performed on the document. This angle information can be a specific angle value or an angle range.

[0068] Specifically, the terminal matches the position state corresponding to each video frame with the target position state, determines whether there is a video frame whose position state matches the target position state, and when the position state corresponding to any video frame successfully matches the target position state, determines the video frame as a candidate key frame to be identified. It can be understood that when there are multiple preset actions, there will also be multiple corresponding target position states, then the terminal can match each video frame with each target position state respectively, and when the position state corresponding to any video frame successfully matches any target position state, determines the video frame as a candidate key frame to be identified. The obtained candidate key frame can be one frame or multiple frames.

[0069] In one embodiment, when there is no video frame whose position state matches the target position state, the terminal may determine that the authenticity authentication of the video frame sequence fails.

[0070] In one embodiment, the position status of the document to be identified in each video frame includes direction information and angle information, and the target position status includes target direction information and target angle information. The terminal matches the direction information corresponding to each video frame with the target direction information, and matches the angle information corresponding to each video frame with the target angle information. When the direction information corresponding to any frame of the video frame is consistent with the target direction information and the angle information corresponding to the video frame is consistent with the target angle information, it means that the position status of the video frame is successfully matched with the target position status, and the terminal can determine the video frame as a candidate key frame.

[0071] Step 210: Determine the authentication result of the document to be authenticated based on the candidate key frames.

[0072] Among them, the identification result includes one of a first identification result and a second identification result. The first identification result is used to indicate that the authenticity identification of the video frame sequence has passed, that is, the document to be identified is a genuine document. The second identification result is used to indicate that the authenticity identification of the video frame sequence has failed, that is, the document to be identified is a fake document or there is no video frame in the video frame sequence whose position state matches the target position state corresponding to the preset action.

[0073] Specifically, since the candidate key frame is a video frame whose position state matches the target position state, if the document to be identified is a genuine document, then its anti-counterfeiting point will change significantly under the target position state, and if the document to be identified is a fake document, then its anti-counterfeiting point will not change under the target position state. Therefore, the terminal can determine the identification result of the document to be identified based on the candidate key frame.

[0074] In one embodiment, there are multiple preset actions, and the target position states of each preset action are obviously different. Therefore, when the terminal matches the position state corresponding to each video frame with the target position state corresponding to the preset action, if the position state corresponding to the video frame matches any target position state, the video frame is determined as the candidate position state of the preset action corresponding to the target position state, and finally, the candidate key frame corresponding to each preset action is obtained. The candidate key frame can be one frame or multiple frames.

[0075] In one embodiment, when there are multiple candidate key frames, the terminal can randomly select one frame from the multiple candidate frames to determine it as the target key frame, thereby obtaining the target key frames corresponding to each candidate action. Based on these target key frames, the terminal can perform authenticity authentication on the document to be authenticated to obtain an authentication result.

[0076] In another embodiment, when there are multiple candidate key frames, the terminal can select the highest-quality frame from the multiple candidate frames as the target key frame, thereby obtaining target key frames corresponding to each candidate action. Based on these target key frames, the terminal can authenticate the document to obtain an authentication result. In a specific embodiment, the highest-quality frame can be, for example, the frame with the highest freshness.

[0077] In the above-mentioned video data processing method, a video frame sequence containing the certificate to be identified is obtained, and the position of the certificate to be identified in the video frame sequence is further determined. Based on the position of the certificate to be identified in the video frame sequence, multiple position states of the certificate to be identified in the video frame sequence are determined, and each position state is matched with a target position state corresponding to a preset action. The video frame corresponding to the successfully matched position state is determined as a candidate key frame to be identified. Since the preset action is an action that causes the anti-counterfeiting point of the standard certificate to change, and the type of the standard certificate is the same as that of the certificate to be identified, the identification result of the certificate to be identified can be determined based on the candidate key frame. In the embodiment of the present application, when authenticating a certificate, only the video frame sequence needs to be obtained, and it does not rely on the certificate entity, which greatly expands the application scope of certificate authenticity identification.

[0078] In one embodiment, determining the position of the document to be identified in the video frame sequence includes: determining an initial video frame from the video frame sequence and obtaining the position of the document to be identified in the initial video frame according to the time sequence of each video frame in the video frame sequence; and tracking the video frames after the initial video frame in the video frame sequence according to the position of the document to be identified in the initial video frame to determine the position of the document to be identified in the video frames after the initial video frame.

[0079] The initial video frame refers to the first video frame in the video frame sequence where the document to be authenticated appears.

[0080] Specifically, the terminal can detect whether the certificate to be identified appears frame by frame starting from the first frame of the video frame sequence according to the time sequence of each video frame in the video frame sequence. When it is detected that the certificate to be identified does not exist in a certain frame, the frame is discarded and the next frame is continued to be detected until the certificate to be identified is detected. The frame is then determined as the initial video frame and the position of the certificate to be identified in the initial video frame is obtained.

[0081] In one embodiment, the terminal can input a video frame sequence frame by frame into an object detection model, which then performs object detection to determine whether the document to be authenticated appears. The object detection model refers to a machine learning model that can be used for object detection. The object detection model can be a machine learning model that performs object detection using a one-stage object detection algorithm, such as the Yolo (You Only Look Once) algorithm or the SSD (Single Shot MultiBox Detector) algorithm, or a machine learning model that performs object detection using a two-stage object detection algorithm, such as Faster RCNN.

[0082] After obtaining the position of the ID to be authenticated in the initial video frame, the terminal performs target tracking on a frame-by-frame basis in the video frame sequence following the initial video frame based on the position to determine the position of the ID to be authenticated in the video frames following the initial video frame. For example, the terminal may perform target tracking on a frame-by-frame basis starting from the frame immediately following the initial video frame to determine the position of the ID to be authenticated in each video frame following the initial video frame.

[0083] Target tracking here refers to tracking the document to be authenticated in the video frames following the initial frame. When performing target tracking, the terminal can use either the Tracking-Learning-Detection (TLD) algorithm or a neural network-based target tracking method. TLD (Tracking-Learning-Detection) is a new single-target, long-term tracking algorithm proposed by Zdenek Kalal of the University of Surrey in the UK. This algorithm combines tracking and detection algorithms to address issues such as deformation and partial occlusion of the target during tracking. Furthermore, an improved online learning mechanism continuously updates the tracking module's "salient feature points" and the detection module's target model and related parameters, making tracking more stable, robust, and reliable. Neural network-based target tracking methods, for example, can be mask-based.

[0084] In the above embodiment, by determining the initial video frame, target tracking can be performed on the video frames after the initial video frame based on the position of the document to be identified in the initial video frame, so that the position of the document to be identified in the video frame sequence can be quickly determined, thereby improving the efficiency of document authentication.

[0085] In one embodiment, Figure 3As shown, according to the time sequence of each video frame in the video frame sequence, determining the initial video frame from the video frame sequence and obtaining the position of the document to be authenticated in the initial video frame includes:

[0086] Step 302: Obtain the trained target detection model.

[0087] Among them, the target detection model includes convolutional layers and fully connected layers.

[0088] Step 304 : determining a current video frame from the video frame sequence according to the time sequence of the video frames in the video frame sequence.

[0089] Specifically, the terminal may select a top-ranked video frame from the video frame sequence according to the time sequence of the video frames in the video frame sequence and determine it as the current video frame. For example, the terminal may determine the first frame in the video frame sequence as the current video frame.

[0090] In step 306, the current video frame is input into the convolution layer, and features of the current video frame are extracted through the convolution layer to obtain a feature map corresponding to the current video frame.

[0091] In step 308, the feature map is input into the fully connected layer, and the document category prediction and position regression are performed through the fully connected layer to obtain the document category prediction result and position corresponding to the current video frame.

[0092] Among them, the document category prediction result is used to represent the category of the target in the initial video frame.

[0093] In one embodiment, the document category prediction result can be a category identifier used to characterize the category to which the target belongs. For example, if there are N possible categories of the target, the first category can be represented by an N-dimensional vector (1, 0, 0, 0, ..., 0), the second category can be represented by a vector (0, 1, 0, 0, ..., 0), and so on.

[0094] In another embodiment, the document category prediction result can be the probability of the category to which the target belongs. For example, if there are N possible categories of the target, then the document category prediction result can be an N-dimensional probability vector, where the probability of each dimension is used to characterize the possibility that the target belongs to that category.

[0095] The position corresponding to the current video frame refers to the position of the target in the current video frame. It can be understood that in the embodiment of the present application, the target is the document to be identified, so the position is the position of the document to be identified.

[0096] Step 310: When the document category prediction result represents the target category, the current video frame is determined as the initial video frame.

[0097] Step 312 : When the document category prediction result represents a non-target category, the current video frame is re-determined from the video frames following the current video frame in the video frame sequence, and the process returns to step 306 .

[0098] The target category refers to the document category required by the current business scenario. Depending on the business scenario, the target category can be one or more, and the category represented by the document category prediction result only needs to belong to one of them.

[0099] It is understandable that in some business scenarios, there are certain requirements for the category of documents. For example, in the application scenario of opening a bank account, the document may be required to be a second-generation ID card. Therefore, when the terminal is tracking the target, it must ensure that the document in the initial video frame is a target category document. Only when the document in the initial video frame is a target category document, the subsequent target tracking steps are performed to ensure the accuracy of target tracking.

[0100] Therefore, if the document category prediction result corresponding to the current video frame is used to represent a document of a non-target category, the terminal discards the video frame and continues to perform object detection on subsequent video frames until a video frame containing a document of the target category is detected and determined as the initial video frame. For example, the terminal may determine the next frame as the current video frame and return to step 306.

[0101] In one embodiment, when the terminal traverses the entire video frame sequence and fails to detect any certificate containing the target certificate category, the current certificate authentication result is determined to be failed.

[0102] In a specific embodiment, when executing step 304, the terminal can determine the first frame in the video frame sequence as the current video frame, and when executing step 312, the terminal can determine the next frame as the current video frame, so that the terminal can perform target detection on the video frames in the video frame sequence frame by frame until a video frame with a target category certificate is detected, and then determine the video frame as the current video frame, thereby ensuring the accuracy of the identification result.

[0103] In a specific embodiment, Figure 10 As shown in Figure 1, it is a schematic diagram of the network structure of the target detection model. Figure 10The target detection model includes three convolutional layers 1000a, 1000b, and 1000c of decreasing sizes, and a fully connected layer 1002. After the current video frame 1004a is input into the target detection model, the convolutional layers 1000a, 1000b, and 1000c extract features from the current video frame 1004a in turn, and input the obtained feature map into the fully connected layer 1002. The fully connected layer 1002 performs document category prediction and position regression, and finally obtains the document category prediction result corresponding to the current video frame 1004a and the position of the document to be identified in the current video frame 1004a. Figure 10 The dotted box in the video frame 1004b is the location of the document to be identified.

[0104] In the above embodiment, a target detection model is used to perform target detection on the video frames in the video frame sequence. Since the target detection model includes a convolutional layer and a fully connected layer, it can accurately extract features and predict the document category and position corresponding to the video frame, thereby improving the efficiency of determining the initial video frame. At the same time, by determining the video frame containing the document category of the target category as the initial video frame, the accuracy of subsequent tracking can be guaranteed.

[0105] In one embodiment, Figure 4 As shown, determining the position of the document to be authenticated in the video frame sequence includes:

[0106] Step 402 : According to the time sequence of each video frame in the video frame sequence, an initial video frame is determined from the video frame sequence and the position of the document to be authenticated in the initial video frame is obtained.

[0107] Step 404 : According to the position corresponding to the initial video frame, an image of the area where the document to be authenticated is located is captured from the initial video frame as a template image.

[0108] Specifically, after the terminal obtains the position of the document to be identified in the initial video frame, it can cut out the image of the area corresponding to the position in the initial video frame according to the position, and use the cut-out image as a template image. The template image only includes the tracking target (i.e., the document to be identified) and can be used as the basis for subsequent target tracking.

[0109] Step 406 : Determine a search graph from video frames subsequent to the initial video frame.

[0110] The video frames after the initial video frame refer to video frames that are later than the initial video frame, and the search graph refers to video frames waiting for target tracking.

[0111] In one embodiment, the terminal may determine the video frames after the initial video frame as a search graph frame by frame, thereby ensuring that the video frame of the document to be authenticated appears in each frame in the video frame sequence is tracked, thereby improving the accuracy of the authentication result.

[0112] In step 408 , the template image and the search image are input into the trained target tracking model, and the target tracking model performs target tracking processing on the search image based on the template image to obtain a tracking result corresponding to the search image.

[0113] Step 410: Determine the position of the document to be authenticated in the search graph based on the tracking result.

[0114] The target tracking model refers to a machine learning model used for target tracking. The machine learning model can perform target tracking processing on the search graph based on the template graph. During the target tracking process, the template graph is used as a basis to track the document to be identified that appears in the search graph. During the tracking process, the terminal can determine the image area in the search graph that is most similar to the tracking target and obtain a tracking result corresponding to the search graph. Based on the tracking result, the terminal can determine the position of the document to be identified in the search graph. In the embodiment of the present application, the tracking target is the document to be identified.

[0115] In one embodiment, the target tracking model can be a Siamese (twin network) series model, for example, the target tracking model can be a siamFC model, a SiamRPN model, a Siammask model, etc. Among them, the siamFC model is a twin network structure based on a fully-convolutional Siamese architecture, and the Siam-RPN is a twin network structure based on an RPN (Region Proposal Network), which consists of a twin network and an RPN network.

[0116] In the above embodiment, by determining a template graph and a search graph, and using a trained target tracking model to perform target tracking processing on the search graph based on the template graph, the efficiency of target tracking can be improved.

[0117] In one embodiment, the trained target tracking model includes a first convolutional layer, a correlation layer, a second convolutional layer, a deconvolution layer, and a refinement layer. In step 408, the template image and the search image are input into the trained target tracking model. The target tracking process of the search image based on the template image by the target tracking model includes the following steps 1-4:

[0118] 1. Input the template image and the search image into the first convolutional layer respectively, and perform feature extraction on the template image and the search image respectively through the first convolutional layer to obtain the template feature map corresponding to the template image and the search feature map corresponding to the search image.

[0119] In one embodiment, the first convolutional layer may include multiple layers of convolution, which perform semantic abstraction on features in the template image and the search image to obtain a template feature map corresponding to the template image and a search feature map corresponding to the search image. The first convolutional layer may employ a common convolutional neural network structure, such as the Alexnet series of networks.

[0120] In one embodiment, the first convolutional layer includes multiple convolutional layers with successively smaller scales. Taking the template feature map corresponding to the template map as an example, the terminal performs multiple feature extractions on the template map through the first convolutional layer. Each time the feature is extracted, the output features of the previous convolutional layer are fused with the input features to obtain the current input features. The current input features are input into the current convolutional layer, and convolution processing is performed through the current convolutional layer. Fusion refers to expressing multiple features with one feature, and the fusion of features can specifically be feature combination, feature splicing, and so on. It can be understood that the process of obtaining the search feature map corresponding to the search map is the same as the process of obtaining the template feature map corresponding to the template map, and will not be repeated here.

[0121] refer to Figure 5 , which is a schematic diagram of the target tracking process using a target tracking model in a specific embodiment. In this embodiment, the first convolutional layer of the target tracking model includes four layers: convolutional layer 502, convolutional layer 504, convolutional layer 506, and convolutional layer 508. The terminal uses these four convolutional layers to perform multiple feature extractions on the template image and the search image. Taking feature extraction of template image A as an example, the terminal first inputs template image A into the convolution layer 502, performs convolution processing through the convolution layer 502 to obtain the output features of the convolution layer 502, and then fuses the output features of the convolution layer 502 with the template image A to obtain the input features of the input convolution layer 504, inputs the input features of the convolution layer 504 into the convolution layer 504, and performs convolution processing through the convolution layer 504 to obtain the output features of the convolution layer 504. Then, the terminal fuses the output features and input features of the convolution layer 504 to obtain the input features of the convolution layer 506, inputs the input features of the convolution layer 506 into the convolution layer 506, and performs convolution processing through the convolution layer 506 to obtain the output features of the convolution layer 506. Further, the terminal fuses the output features and input features of the convolution layer 506 to obtain the input features of the convolution layer 508, and inputs the input features of the convolution layer 508 into the convolution layer 508 to obtain the template feature image A1 corresponding to the template image A. Similarly, the terminal can extract features from the search graph B through convolution layer 502, convolution layer 504, convolution layer 506, and convolution layer 508 to obtain a corresponding search feature graph B1, which will not be repeated here.

[0122] 2. Input the template feature map and the search feature map into the correlation layer, and perform cross-correlation on the template feature map and the search feature map through the correlation layer to obtain a correlation feature map; the correlation feature map includes multiple candidate window responses.

[0123] Cross-correlation refers to performing a correlation convolution operation within a specific quantized range. The correlation feature map includes multiple candidate window responses, each describing the similarity response of each region element in the search map to the target. The larger the response value, the more similar the image in that region is to the target, and the more likely it is the region where the target appears. Obviously, the region with the largest response value is the region where the target appears.

[0124] Specifically, the terminal inputs the template feature map and the search feature map into the relevant layer, performs channel-by-channel cross-correlation convolution on the template feature map and the search feature map through the relevant layer to obtain a relevant feature map, and performs subsequent tracking processing steps through the relevant feature map.

[0125] Continue to refer Figure 5 After the terminal inputs the template feature map and the search feature map into the relevant layer (not shown in the figure), the relevant feature map C is obtained.

[0126] 3. Input the relevant feature map into the second convolutional layer to obtain the score feature map and mask feature map respectively.

[0127] Continue to refer Figure 5 , the terminal inputs the relevant feature map C into the second convolutional layer (not shown in the figure) to obtain the score feature map D and the mask feature map E.

[0128] 4. Determine the candidate mask feature subgraph from the mask feature graph according to the scores corresponding to each mask feature subgraph, input the candidate mask feature subgraph into the deconvolution layer, and perform deconvolution processing on the candidate mask feature subgraph through the deconvolution layer to obtain the target mask feature subgraph.

[0129] The second convolutional layer consists of two parallel 1x1 convolutional layers. Each convolutional layer performs different channel processing on the relevant feature maps, generating score feature maps and mask feature maps. The mask feature map includes the mask feature submap corresponding to each candidate window response, and the score feature map includes the score corresponding to each mask feature submap. The mask feature submap (mask) is a binary pixel map. A higher score for a mask feature submap indicates a higher response value for the candidate window response corresponding to that mask feature submap.

[0130] Specifically, there is a positional correspondence between the mask feature map and the score feature map. The terminal can determine the position index with the highest score based on the scores corresponding to each mask feature submap, determine the corresponding mask feature submap from the mask feature map based on the position index, and determine the mask feature submap as a candidate mask feature submap. For example, if the terminal determines that the position with the highest score is in the second row and third column based on the scores corresponding to each mask feature submap, the terminal will correspondingly determine the mask feature submap in the second row and third column of the mask feature map as a candidate mask feature submap.

[0131] The terminal then feeds the candidate mask feature subgraphs into the deconvolution layer, which performs a deconvolution operation on them to increase their size and obtain the target mask feature subgraph. Deconvolution is a special form of forward convolution that first increases the size of the input image by padding it with zeros, then rotates the convolution kernel before performing the forward convolution again.

[0132] Continue to refer Figure 5 , the terminal selects the mask feature subgraph F with the highest score from the mask feature graph according to the score in the score feature graph. The terminal further inputs the mask feature subgraph F into the deconvolution layer for deconvolution processing to obtain the target mask feature subgraph G.

[0133] 5. Input the target mask feature sub-image into the refinement layer, and refine the target mask feature sub-image through the refinement layer to obtain the mask image corresponding to the search image.

[0134] Among them, the refinement process is used to combine the intermediate feature information obtained during feature extraction during upsampling. The intermediate features here refer to the output features of the convolution layer before the last convolution layer when extracting features through multi-layer convolution. For example, continue to refer to Figure 5 In the first convolutional layer, convolutional layers 502, 504, and 506 all output intermediate features. In the mask corresponding to the search image, the pixels in the area where the ID to be identified are set to 1, while the pixels in the background area are set to 0, thereby segmenting the ID to be identified from the search image. Through refinement, the model's high-level semantics and low-level features are combined, resulting in a more accurate mask.

[0135] In one embodiment, the first convolution layer includes multiple convolution layers with successively smaller scales, and the refinement layer includes multiple convolution layers with successively larger scales. When the terminal extracts features from the search image, it can perform multiple feature extractions on the search image through the first convolution layer, and save multiple intermediate features obtained during the feature extraction process. Thus, when the target mask feature sub-image is input into the refinement layer and the target mask feature sub-image is refined through the refinement layer, the target mask feature is upsampled multiple times through the refinement layer. Each time the image is upsampled, the output feature of the previous convolution layer is fused with the corresponding intermediate feature to obtain the current input feature. The current input feature is input into the current convolution layer, and deconvolution is performed through the current convolution layer. After multiple upsamplings, a mask image with a size that meets the requirements is finally obtained.

[0136] Continue to refer Figure 5 The refinement layer includes four convolutional layers 510, 512, 514, and 516 with increasing scales. The terminal first inputs the target mask feature subgraph G into the convolutional layer 510, performs deconvolution processing through the convolutional layer 510, obtains the output features of the convolutional layer 510, fuses the output features of the convolutional layer 510 with the output features of the convolutional layer 506, obtains the input features of the convolutional layer 512, inputs the input features of the convolutional layer 512 into the convolutional layer 512, performs deconvolution processing through the convolutional layer 512, obtains the output features of the convolutional layer 512, and fuses the output features of the convolutional layer 512 with the output features of the convolutional layer 506. The output features of the convolution layer 504 are fused to obtain the input features of the convolution layer 514, the input features of the convolution layer 514 are input to the convolution layer 514, and the convolution layer 514 performs deconvolution processing to obtain the output features of the convolution layer 514, the output features of the convolution layer 514 are fused with the output features of the convolution layer 502 to obtain the input features of the convolution layer 516, the input features of the convolution layer 516 are input to the convolution layer 516, and the convolution layer 516 performs deconvolution processing to obtain the mask image H.

[0137] In one embodiment, the above-mentioned step 206 determines multiple position states of the document to be identified in the video frame sequence based on the position of the document to be identified in the video frame sequence, including: filtering video frames in the video frame sequence that do not meet preset conditions based on the position of the document to be identified in the video frame sequence; inputting the position of the document to be identified in the filtered video frame sequence into a trained position state prediction network; and determining the position state of the document to be identified in the video frame sequence based on the output result of the position state prediction network.

[0138] Preset conditions refer to the conditions used to filter out video frames that do not meet the standards. For example, a preset condition could be that the ID to be authenticated in the video frame cannot exceed the frame boundary, or that the ID to be authenticated's proportion of the video frame cannot be less than a preset threshold. Preset conditions can be set based on the needs of the scenario.

[0139] The position state prediction network is used to predict the position state category corresponding to the video frame. Position state categories include: dividing the process of flipping the document upward into multiple angle intervals to obtain multiple up-flip categories; dividing the process of flipping the document downward into multiple angle intervals to obtain multiple down-flip categories; dividing the process of flipping the document leftward into multiple angle intervals to obtain multiple left-flip categories; dividing the process of flipping the document rightward into multiple angle intervals to obtain multiple down-flip categories; and obtaining a flat category for the document being laid flat. For example, position state categories may include: up-flip 20-40, up-flip 40-60, up-flip 60-80, down-flip 20-40, down-flip 40-60, down-flip 60-80, left-flip 20-40, left-flip 40-60, left-flip 60-80, right-flip 20-40, right-flip 40-60, right-flip 60-80, and flat (which can be viewed as an angle interval of -20-20).

[0140] In one embodiment, the position state prediction network is trained by obtaining a training sample, the training sample including a video frame containing a document to be identified and a position state category label to which the document to be identified in the video frame belongs, using the video frame as the input of the position state prediction network, obtaining the actual output of the position state prediction network, adjusting the parameters of the position state prediction network based on the difference between the actual output and the position state category label, and terminating the training until a training stop condition is met, thereby obtaining a trained position state prediction network. The training stop condition may be that the difference is less than a preset threshold, the number of training times reaches a preset number, the training duration reaches a preset duration, etc. During the training process, a stochastic gradient algorithm may be used to calculate the loss gradient, and the loss gradient may be back-propagated to adjust the network parameters of the position state prediction network.

[0141] Specifically, the terminal determines whether each video frame satisfies a preset condition based on the position of the identification document within each frame in the video frame sequence, discards any video frames that do not meet the preset condition, and thereby filters out video frames in the video frame sequence that do not meet the preset condition. Furthermore, the terminal inputs the position corresponding to each frame in the filtered video frame sequence into a trained position state prediction network. The position state prediction network performs position state prediction, obtaining a corresponding output result. Based on this output result, the position state of the identification document within each frame in the video frame sequence is determined. The output result here can be a category identifier representing a position state category, or a probability representing a position state category.

[0142] In the above embodiment, by filtering the video frame sequence, video frames that do not meet the preset conditions can be filtered out to ensure the accuracy of the identification results. At the same time, the position state can be quickly obtained by inputting the position into the position state prediction network, thereby ensuring the accuracy and efficiency of document authenticity identification.

[0143] In one embodiment, determining the identification result of the document to be identified based on the candidate key frames in the above step 210 includes: obtaining the quality score of each candidate key frame; determining the target key frame based on the quality score of each candidate key frame; and determining the identification result of the document to be identified based on the target key frame.

[0144] The quality score is used to characterize the clarity of the candidate key frame. The higher the quality score, the clearer the candidate video frame is and the better the identification effect is.

[0145] Specifically, in this embodiment, the candidate key frames include multiple frames, each of which contains sufficient information about the document's security features. However, some security features are not readily apparent. Therefore, to highlight these features, the terminal needs to select a clear candidate key frame from the multiple candidate video frames and determine it as the target key frame. In one specific embodiment, the terminal may select the video frame with the highest quality score as the target key frame.

[0146] In one embodiment, the terminal can evaluate the image quality of each candidate key frame using a deep learning algorithm to obtain a quality score. This quality score is the absolute quality score of the candidate key frame. Specifically, the terminal can obtain training sample images and corresponding quality score labels, train a quality assessment model based on the training sample images and corresponding quality score labels, and input each candidate key frame into the trained quality assessment model to obtain a quality score corresponding to each candidate key frame.

[0147] In another embodiment, since all candidate key frames are from the same video, the target key frame can be determined by comparing the relative quality scores of the candidate video frames. Based on this, in a specific embodiment, obtaining the quality scores of the candidate key frames may include the following steps 1-3:

[0148] 1. Get the horizontal and vertical gradients of each color channel corresponding to the candidate keyframe.

[0149] 2. Obtain the gradient value of each color channel corresponding to the candidate key frame based on the horizontal gradient and vertical gradient of each color channel corresponding to the candidate key frame.

[0150] Specifically, the terminal can use edge detection operators such as the Canny operator and the Sobel operator to perform edge detection on candidate keyframes, calculate the horizontal and vertical gradients of each color channel, and then obtain the gradient value corresponding to each color channel based on the horizontal and vertical gradients. Obviously, for a set of candidate frames for the same keyframe, the richer the edge information, the clearer the frame and the higher the quality. It can be understood that the color channels here refer to R (red), G (green), and B (blue). The horizontal gradient refers to the rate of change of a pixel in the image on the x-axis (compared to adjacent pixels), and the vertical gradient refers to the rate of change of a pixel in the image on the y-axis.

[0151] Taking the Sobel operator as an example, the terminal can calculate the lateral gradient by the following formula (1): , and the longitudinal gradient is calculated by the following formula (2) , and calculate the gradient value by the following formula (3) , where A is the candidate video frame:

[0152] Formula (1)

[0153] Formula (1)

[0154] Formula (3)

[0155] In specific implementation, for each candidate key frame, since the key frame of the authentication video only focuses on the clarity of the main part of the document to be authenticated, the terminal can crop out the document body of each candidate key frame based on the previous target tracking results to eliminate background interference and perform edge detection on the cropped document body.

[0156] 3. Accumulate the gradient values ​​of each color channel to obtain the quality score of the candidate key frame.

[0157] Specifically, after obtaining the gradient values ​​of the R channel, the G channel, and the B channel through the above steps, the terminal sums the gradient values ​​of the R channel, the G channel, and the B channel to obtain the quality score of the candidate key frame.

[0158] In the above embodiment, the target key frame is determined by calculating the relative quality score. Compared with the deep learning method, the amount of calculation is small, thereby improving the identification efficiency.

[0159] In one embodiment, Figure 6 As shown, the above step 210 determines the authentication result of the document to be authenticated based on the candidate key frames, including:

[0160] Step 602: Obtain the quality score of each candidate key frame.

[0161] Step 604: respectively form candidate key frame sets corresponding to each preset action from the candidate key frames corresponding to each preset action.

[0162] Specifically, in this embodiment, there are multiple preset actions, different candidate key frames correspond to each preset action, and multiple candidate key frames correspond to each preset action respectively. The terminal respectively forms candidate key frame sets corresponding to each preset action from the candidate key frames corresponding to each preset action.

[0163] Step 606: respectively determine target key frames of each preset action corresponding to each candidate key frame set according to the quality scores of the candidate key frames in each candidate key frame set.

[0164] Specifically, the terminal respectively selects the candidate key frame with the highest quality score from each candidate key frame set as the target key frame of each preset action corresponding to each candidate key frame set.

[0165] Step 608: perform channel splicing processing on each target key frame to obtain a spliced key frame.

[0166] Specifically, the terminal performs channel splicing processing on each target key frame to obtain a spliced key frame. The channel splicing processing here refers to horizontal splicing according to channels.

[0167] Illustrate with an example. Suppose the candidate key frames include X1, X2, X3, Y1, Y2, Y3, Y4, Z1, Z2, and the quality scores corresponding to these candidate key frames are a1, a2, a3, a4, a5, a6, a7, a8, a9 respectively. Suppose there are three preset actions: turn up 30 degrees, lay flat, and turn down 30 degrees. Among them, X1, X2, X3 are candidate key frames corresponding to turning up 30 degrees, forming a candidate key frame set {X1, X2, X3} corresponding to turning up 30 degrees and a1 < a2 < a3, Y1, Y2, Y3, Y4 are candidate key frames corresponding to laying flat, forming a candidate key frame set {Y1, Y2, Y3, Y4} corresponding to laying flat and a4 < a5 < a6 < a7, Z1, Z2 are candidate key frames corresponding to laying flat, forming a candidate key frame set {Z1, Z2} corresponding to laying flat and a8 < a9. Then the terminal determines that the target key frame corresponding to turning up 30 is X3, the target key frame corresponding to laying flat is Y4, and the target key frame corresponding to turning down 30 is Z2. Suppose the channel vector corresponding to the target key frame X3 is (R1, G1, B1), the channel vector corresponding to the target key frame Y4 is (R2, G2, B2), and the channel vector corresponding to the target key frame Z2 is (R3, G3, B3). Then the channel vector corresponding to the spliced key frame obtained after splicing is (R1, G1, B1, R2, G2, B2, R3, G3, B3).

[0168] In step 610, the spliced ​​key frame is input into the trained classification network, and the identification result of the document to be identified is determined according to the classification result output by the classification network.

[0169] The identification result classification network is used to classify the identification results. The output of the identification result classification network can be an identification result identifier. For example, 1 or the vector (1,0) represents the first identification result, and 0 or the vector (0,1) represents the second identification result. The terminal can then obtain the identification result of the document to be identified based on the identification result identifier. The output of the identification result classification network can also be a probability vector (a%, b%), where a% is the probability of the first identification result and b% is the probability of the second identification result. The terminal can then determine the identification result with the highest probability.

[0170] In the above embodiment, since the identification result is obtained by performing channel splicing based on multiple target key frames, it is more robust and more accurate than related technologies.

[0171] In one embodiment, each position state includes direction information and angle information; the target position state includes target direction information and target angle information; each position state is matched with the target position state corresponding to the preset action, and the video frame corresponding to the successfully matched position state is determined as the candidate key frame to be identified, including: matching the direction information corresponding to each position state with the target direction information, and matching the angle information corresponding to each position state with the target angle information; determining the position state whose direction information is consistent with the target direction information and whose angle information is consistent with the target angle information as the successfully matched position state.

[0172] The direction information refers to the direction in which the document to be authenticated is flipped. This flipping direction can include upward, downward, left, or right. The angle information refers to the specific angle at which the document is flipped, or the angle range to which the angle belongs. For example, the angle information can be 30 degrees or 20-50 degrees.

[0173] Specifically, the terminal matches the direction information corresponding to the position state with the target direction information, and matches the angle information corresponding to each position state with the target angle information. When the direction information corresponding to any position state is consistent with the target direction information and the angle information corresponding to the position state is consistent with the target angle information, it is determined that the position state is successfully matched with the target position state, and the terminal determines the video frame corresponding to the position state as the candidate key frame to be identified.

[0174] In one embodiment, the target angle information is an angle range, and the angle information in the position state corresponding to the candidate keyframe is a specific angle value. The terminal determines whether the angle value is within the angle range corresponding to the target angle information. If so, the angle information corresponding to the position state is determined to be consistent with the target angle information. For example, if the angle information in the position information corresponding to candidate keyframe X is 30 degrees and the target angle information is between 20 and 50 degrees, the angle information corresponding to the position state can be determined to be consistent with the target angle information. If the angle information in the position information corresponding to candidate keyframe X is 70 degrees, the angle information corresponding to the position state can be determined to be inconsistent with the target angle information.

[0175] In another embodiment, the target angle information is a target angle interval, the angle information in the position state corresponding to the candidate key frame is an angle interval, and the position state in which the direction information is consistent with the target direction information and the angle information is consistent with the target angle information is determined as a successfully matched position state, including: when the angle interval corresponding to any position state is included in the target angle interval corresponding to the target position state, it is determined that the angle information corresponding to the position state is consistent with the target angle information.

[0176] Here, angle interval A is included in angle interval B means that the minimum value of angle interval A is not less than the minimum value of angle interval B and the maximum value of angle interval A is not greater than the maximum value of angle interval B. For example, if angle interval A is 40 degrees to 50 degrees and angle interval B is 20 degrees to 60 degrees, then angle interval A is included in angle interval B. If angle interval A is 40 degrees to 50 degrees and angle interval B is 45 degrees to 70 degrees, then angle interval A is not included in angle interval B.

[0177] Specifically, the terminal can compare the angle intervals contained in the position states corresponding to each video frame with the target angle interval respectively. That is, when the angle interval corresponding to a position state is included in the target angle interval corresponding to the target position state, it is determined that the angle information corresponding to the position state is consistent with the target angle information, and the terminal determines the video frame corresponding to the position state as the candidate key frame to be identified.

[0178] like Figure 7 As shown in FIG. 1 , a step of obtaining a target key frame by a terminal in a specific embodiment is shown. Figure 7After acquiring the video frame sequence, the terminal determines frame by frame whether the target category certificate is detected starting from the first frame of the video frame sequence. If not, the frame is discarded and the judgment is continued on the next frame. If so, the target tracking is started to obtain the certificate position in each frame, and further judges whether the anti-counterfeiting point of the certificate has changed significantly based on the position of the certificate in each frame (see the description in the above embodiment for the specific judgment process). If not, the frame is discarded; if so, the frame is retained as a candidate key frame. Further, the terminal selects the frame with the best quality from the candidate key frames as the target key frame. Further, the terminal determines whether a preset number of target key frames are obtained. If so, these target key frames are output. If not, the video frame sequence is continued to be acquired and the above process is repeated.

[0179] In a specific embodiment, a video data processing method is provided, comprising the following steps:

[0180] 1. The terminal obtains a sequence of video frames containing the document to be authenticated.

[0181] 2. Obtain the trained target detection model; the target detection model includes convolutional layers and fully connected layers.

[0182] 3. Determine the current video frame from the video frame sequence according to the time sequence of each video frame in the video frame sequence.

[0183] 4. Input the current video frame into the convolutional layer, extract the features of the current video frame through the convolutional layer, obtain the feature map corresponding to the current video frame, input the feature map into the fully connected layer, perform document category prediction and position regression through the fully connected layer, and obtain the document category prediction result and position corresponding to the current video frame.

[0184] 5. When the document category prediction result corresponding to the current video frame is used to represent the target category document, the current video frame is determined as the initial video frame.

[0185] 6. When the document category prediction result corresponding to the current video frame is used to represent a non-target category document, the current video frame is re-determined from the video frames following the current video frame in the video frame sequence and the process returns to step 4.

[0186] 7. Obtain the trained target tracking model, which includes a first convolutional layer, a correlation layer, a second convolutional layer, a deconvolution layer, and a refinement layer.

[0187] 8. According to the position corresponding to the initial video frame, an image of the area where the document to be identified is located is captured from the initial video frame as a template image, and a search image is determined from the video frames after the initial video frame. For each search image, the terminal executes the following steps 9-14 to determine the position of the document to be identified.

[0188] 9. Input the template image and the search image into the first convolutional layer respectively, perform feature extraction on the template image and the search image respectively through the first convolutional layer to obtain the template feature image corresponding to the template image and the search feature image corresponding to the search image.

[0189] The first convolutional layer includes multiple layers of convolutional layers with successively smaller scales.

[0190] Specifically, the template image is subjected to multiple feature extractions through the first convolutional layer. Each time feature extraction is performed, the output features of the previous convolutional layer are fused with the input features to obtain the current input features, which are then input into the current convolutional layer for convolution processing.

[0191] The search graph is subjected to multiple feature extractions through the first convolutional layer. Each time a feature is extracted, the output features of the previous convolutional layer are fused with the input features to obtain the current input features. The current input features are input into the current convolutional layer, and convolution processing is performed through the current convolutional layer to save multiple intermediate features obtained during the feature extraction process.

[0192] 10. Input the template feature map and the search feature map into the correlation layer, and perform cross-correlation on the template feature map and the search feature map through the correlation layer to obtain a correlation feature map; the correlation feature map includes multiple candidate window responses.

[0193] 11. Input the relevant feature map into the second convolutional layer to obtain a score feature map and a mask feature map respectively; the mask feature map includes the mask feature sub-map corresponding to each candidate window response, and the score feature map includes the scores corresponding to each mask feature sub-map.

[0194] 12. Determine the candidate mask feature subgraph from the mask feature graph according to the scores corresponding to each mask feature subgraph, input the candidate mask feature subgraph into the deconvolution layer, and perform deconvolution processing on the candidate mask feature subgraph through the deconvolution layer to obtain the target mask feature subgraph.

[0195] 13. Input the target mask feature sub-image into the refinement layer, and refine the target mask feature sub-image through the refinement layer to obtain the mask image corresponding to the search image.

[0196] Among them, the refinement layer includes multiple layers of convolutional layers with successively larger scales.

[0197] Specifically, the target mask features are upsampled multiple times through the refinement layer. Each time the features are upsampled, the output features of the previous convolution layer are fused with the corresponding intermediate features to obtain the current input features. The current input features are input into the current convolution layer, and deconvolution processing is performed through the current convolution layer.

[0198] 14. Determine the location of the document to be identified in the search image based on the mask image.

[0199] Specifically, the terminal may perform straight line detection on the mask image to obtain four straight lines, obtain the coordinates of the intersections between the four straight lines, and obtain the position of the document to be identified in the search image.

[0200] 15. According to the position of the certificate to be identified in the video frame sequence, the video frames in the video frame sequence that do not meet the preset conditions are filtered.

[0201] 16. Input each position of the document to be identified in the filtered video frame sequence into the trained position state prediction network, and determine the position state of the document to be identified in the video frame sequence according to the output result of the position state prediction network.

[0202] 17. Each position state is matched with the target position state corresponding to a preset action, and the video frame corresponding to the successfully matched position state is determined as the candidate key frame to be identified; the preset action is an action that causes the anti-counterfeiting point of the standard document to change, and the standard document and the document to be identified are of the same type. There are multiple preset actions.

[0203] Each position state includes direction information and angle information; the target position state includes target direction information and target angle information.

[0204] Specifically, the terminal matches the direction information corresponding to each position state with the target direction information, and matches the angle information corresponding to each position state with the target angle information; a position state in which the direction information is consistent with the target direction information and the angle information is consistent with the target angle information is determined to be a successfully matched position state. The angle information is an angle interval; the target angle information is a target angle interval; when the angle interval corresponding to any position state is included in the target angle interval corresponding to the target position state, the angle information corresponding to the position state is determined to be consistent with the target angle information.

[0205] 18. Obtain a quality score for each candidate key frame; the quality score is used to characterize the clarity of the candidate key frame.

[0206] Specifically, the terminal obtains the horizontal gradient and vertical gradient of each color channel corresponding to the candidate key frame respectively, obtains the gradient value of each color channel corresponding to the candidate key frame according to the horizontal gradient and vertical gradient of each color channel corresponding to the candidate key frame, accumulates the gradient value of each color channel, and obtains the quality score of the candidate key frame.

[0207] 19. The candidate key frames corresponding to the preset actions are respectively grouped into candidate key frame sets corresponding to the preset actions, and the target key frames of the preset actions corresponding to the candidate key frame sets are determined according to the quality scores of the candidate key frames in the candidate key frame sets.

[0208] 20. Perform channel splicing processing on each target key frame to obtain a spliced ​​key frame, and input the spliced ​​key frame into the trained identification result classification network to obtain the identification result of the document to be identified.

[0209] This application also provides an application scenario, which applies the above-mentioned video data processing method. Specifically, the application of the video data processing method in this application scenario is as follows:

[0210] In this application scenario, the terminal obtains the trained target detection model, target tracking model, position state prediction network, and identification result classification network from other computer devices and stores them locally.

[0211] In this application scenario, the user opens an e-banking account through the terminal. During the account opening process, the terminal authenticates the ID card information provided by the user. The terminal can prompt the user to shoot a video holding the ID card. During the video shooting process, the user is prompted by voice prompts to perform three preset actions on the ID card. After the user completes the shooting, the terminal obtains a video frame sequence, and inputs the target detection model frame by frame starting from the first frame of the video frame sequence for target detection. When the ID card is detected, the subsequent frames are tracked according to the position coordinates of the identity frame to obtain the position coordinates of the four vertices of the ID card in each frame of the video. During the tracking process, the terminal determines the video frame where the ID card is detected as the template image, and determines the video frames after the template image as the search image frame by frame. For each search image, the terminal inputs the search image and the template image into the target tracking model to obtain the tracking results corresponding to each search image, and determines the position coordinates of the four vertices of the ID card in the search image based on the tracking results.

[0212] Furthermore, the terminal inputs the position coordinates of the four vertices of the ID card in each frame of video into the trained position state prediction network, determines the position state of the document to be identified in the video frame sequence based on the output result of the position state prediction network, matches the position state corresponding to each frame of video with the target position state corresponding to each of the three preset actions, and determines the video frames that successfully match the target position state corresponding to each preset action as the candidate key frames corresponding to each preset action. The relative quality score of each candidate key frame is calculated by the edge detection algorithm, and the candidate key frame with the highest relative quality score is selected from the candidate key frames corresponding to each preset action as the target key frame corresponding to each preset action. The target key frames are channel-spliced ​​to obtain spliced ​​key frames, and the spliced ​​key frames are input into the trained identification result classification network to obtain the identification result of the document to be identified. When the identification result is that the authenticity identification passes, the terminal continues to execute the subsequent account opening process. When the identification result is that the authenticity identification fails, a prompt message is given to terminate the account opening process.

[0213] This application also provides another application scenario, which applies the above-mentioned video data processing method. Specifically, the application of the video data processing method in this application scenario is as follows:

[0214] In this application scenario, users need to perform real-name authentication in the social application of the terminal. During the real-name authentication process, the user selects a video frame sequence containing the ID card from the album and uploads the video frame sequence to the server in the social application. The server stores the trained target detection model, target tracking model, location status prediction network and identification result classification network.

[0215] After receiving the video frame sequence, the server inputs each frame, starting with the first frame, into the target detection model for target detection. When an ID card is detected, the server tracks the target in subsequent frames based on the position coordinates of the ID card frame to obtain the position coordinates of the four vertices of the ID card in each frame. During the tracking process, the server identifies the video frame in which the ID card is detected as a template graph and then identifies each frame following the template graph as a search graph. For each search graph, the server inputs the search graph and the template graph into the target tracking model, obtains the tracking results corresponding to each search graph, and then determines the position coordinates of the four vertices of the ID card in the search graph based on the tracking results.

[0216] Furthermore, the server inputs the position coordinates of the four vertices of the ID card in each frame of video into the trained position state prediction network, determines the position state of the ID card to be identified in the video frame sequence according to the output result of the position state prediction network, matches the position state corresponding to each frame of video with the target position state corresponding to each preset action, and determines the video frames that successfully match the target position state corresponding to each preset action as the candidate key frames corresponding to each preset action, calculates the relative quality score of each candidate key frame through the edge detection algorithm, and selects the candidate key frame with the highest relative quality score from the candidate key frames corresponding to each preset action to determine it as each The target key frames corresponding to the preset actions are channel-spliced ​​to obtain spliced ​​key frames, and the spliced ​​key frames are input into the trained identification result classification network to obtain the identification result of the document to be identified. When the server determines that the authenticity identification is passed according to the identification result, the identification result of passing is returned to the terminal. After receiving the identification result, the terminal extracts the ID card information from the video frame sequence, including the ID card number and ID card portrait, and binds the ID card information with the user's social account to achieve real-name authentication; when the server determines that the authenticity identification is failed according to the identification result, the identification result of failing is returned to the terminal, and the terminal prompts the user that the real-name authentication has failed.

[0217] It should be understood that although Figure 1-7The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1-7 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The order of execution of these steps or stages is not necessarily one by one, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0218] In one embodiment, Figure 8 As shown, a video data processing device 800 is provided. The device can be a software module or a hardware module, or a combination of both to form a part of a computer device. The device specifically includes: a video frame acquisition module 802, a position determination module 804, a position state determination module 806, a matching module 808 and an identification result determination module 810, wherein:

[0219] The video frame acquisition module 802 is used to acquire a video frame sequence containing the document to be authenticated;

[0220] A position determination module 804 is used to determine the position of the document to be authenticated in the video frame sequence;

[0221] A position state determination module 806 is configured to determine a plurality of position states of the document to be identified in the video frame sequence according to the position of the document to be identified in the video frame sequence;

[0222] Matching module 808 is configured to match each position state with a target position state corresponding to a preset action, and determine the video frame corresponding to the successfully matched position state as a candidate key frame to be authenticated; the preset action is an action that causes the anti-counterfeiting point of the standard document to change, and the standard document and the document to be authenticated are of the same type;

[0223] The identification result determination module 810 is used to determine the identification result of the document to be identified based on the candidate key frames.

[0224] In one embodiment, the position determination module 804 is used to determine the initial video frame from the video frame sequence and obtain the position of the document to be identified in the initial video frame based on the time sequence of each video frame in the video frame sequence; based on the position of the document to be identified in the initial video frame, the video frames after the initial video frame in the video frame sequence are tracked to determine the position of the document to be identified in the video frames after the initial video frame.

[0225] In one embodiment, the position determination module 804 is also used to obtain a trained target detection model; the target detection model includes a convolutional layer and a fully connected layer; according to the time sequence of each video frame in the video frame sequence, the current video frame is determined from the video frame sequence; the current video frame is input into the convolutional layer, and the feature of the current video frame is extracted through the convolutional layer to obtain a feature map corresponding to the current video frame; the feature map is input into the fully connected layer, and the document category prediction and position regression are performed through the fully connected layer to obtain the document category prediction result and position corresponding to the current video frame; when the document category prediction result corresponding to the current video frame is used to represent the target category, the current video frame is determined as the initial video frame; when the document category prediction result corresponding to the current video frame is used to represent the non-target category, the current video frame is re-determined from the video frames after the current video frame in the video frame sequence and the step of inputting the current video frame into the convolutional layer is returned.

[0226] In one embodiment, the position determination module 804 is further used to determine the initial video frame from the video frame sequence and obtain the position of the document to be identified in the initial video frame according to the time sequence of each video frame in the video frame sequence; intercept the image of the area where the document to be identified is located from the initial video frame according to the position corresponding to the initial video frame as a template image; determine the search image from the video frames after the initial video frame; input the template image and the search image into the trained target tracking model, and perform target tracking processing on the search image based on the template image through the target tracking model to obtain a tracking result corresponding to the search image; and determine the position of the document to be identified in the search image according to the tracking result.

[0227] In one embodiment, the trained target tracking model includes a first convolution layer, a correlation layer, a second convolution layer, a deconvolution layer, and a refinement layer; the position determination module 804 is further used to input the template image and the search image into the first convolution layer respectively, perform feature extraction on the template image and the search image respectively through the first convolution layer, and obtain a template feature image corresponding to the template image and a search feature image corresponding to the search image; input the template feature image and the search feature image into the correlation layer, perform cross-correlation on the template feature image and the search feature image through the correlation layer, and obtain a correlation feature image; the correlation feature image includes multiple candidate window responses; input the correlation feature image into the correlation layer; The second convolutional layer obtains a score feature map and a mask feature map respectively; the mask feature map includes the mask feature submap corresponding to each candidate window response, and the score feature map includes the score corresponding to each mask feature submap; according to the score corresponding to each mask feature submap, the candidate mask feature submap is determined from the mask feature map, and the candidate mask feature submap is input into the deconvolution layer, and the candidate mask feature submap is deconvolved through the deconvolution layer to obtain the target mask feature submap; the target mask feature submap is input into the refinement layer, and the target mask feature submap is refined through the refinement layer to obtain the mask map corresponding to the search image.

[0228] In one embodiment, the first convolutional layer includes multiple layers of convolutional layers with successively decreasing scales; the position determination module 804 is further used to perform multiple feature extractions on the template image through the first convolutional layer. Each time feature extraction is performed, the output features of the previous convolutional layer are fused with the input features to obtain the current input features, and the current input features are input into the current convolutional layer for convolution processing through the current convolutional layer.

[0229] In one embodiment, the first convolutional layer includes multiple convolutional layers with successively decreasing scales; the refinement layer includes multiple convolutional layers with successively increasing scales; the position determination module 804 is further used to perform multiple feature extractions on the search image through the first convolutional layer, and save multiple intermediate features obtained during the feature extraction process; the target mask feature sub-image is upsampled multiple times through the refinement layer, and each time the upsampling is performed, the output feature of the previous convolutional layer is fused with the corresponding intermediate feature to obtain the current input feature, the current input feature is input into the current convolutional layer, and deconvolution processing is performed through the current convolutional layer.

[0230] In one embodiment, the position state determination module 806 is used to filter the video frames in the video frame sequence that do not meet the preset conditions based on the position of the certificate to be identified in the video frame sequence; input the position of the certificate to be identified in the filtered video frame sequence into the trained position state prediction network; and determine the position state of the certificate to be identified in the video frame sequence based on the output result of the position state prediction network.

[0231] In one embodiment, the candidate key frames include multiple frames; the identification result determination module 810 is used to obtain the quality score of each candidate key frame; the quality score is used to characterize the clarity of the candidate key frame; the target key frame is determined based on the quality score of each candidate key frame; and the identification result of the document to be identified is determined based on the target key frame.

[0232] In one embodiment, the identification result determination module 810 is used to obtain the horizontal gradient and vertical gradient of each color channel corresponding to the candidate key frame; obtain the gradient value of each color channel corresponding to the candidate key frame based on the horizontal gradient and vertical gradient of each color channel corresponding to the candidate key frame; accumulate the gradient values ​​of each color channel to obtain the quality score of the candidate key frame.

[0233] In one embodiment, the preset actions include multiple ones, and each preset action corresponds to a different candidate key frame; the identification result determination module 810 is used to respectively group the candidate key frames corresponding to each preset action into a candidate key frame set corresponding to each preset action; determine the target key frame of the preset action corresponding to each candidate key frame set according to the quality scores of the candidate key frames in each candidate key frame set; perform channel splicing processing on each target key frame to obtain a spliced ​​key frame; input the spliced ​​key frame into the trained identification result classification network to obtain the identification result of the document to be identified.

[0234] In one embodiment, each position state includes direction information and angle information; the target position state includes target direction information and target angle information; the matching module 808 is used to match the direction information corresponding to each position state with the target direction information, and to match the angle information corresponding to each position state with the target angle information; the position state in which the direction information is consistent with the target direction information and the angle information is consistent with the target angle information is determined as a position state in which the matching is successful.

[0235] In one embodiment, the angle information is an angle interval; the target angle information is a target angle interval; the matching module 808 is also used to determine that the angle information corresponding to the position state is consistent with the target angle information when the angle interval corresponding to any position state is included in the target angle interval corresponding to the target position state.

[0236] The above-mentioned video data processing device obtains a video frame sequence containing the certificate to be identified, and further determines the position of the certificate to be identified in the video frame sequence. Based on the position of the certificate to be identified in the video frame sequence, it determines multiple position states of the certificate to be identified in the video frame sequence, matches each position state with the target position state corresponding to the preset action, and determines the video frame corresponding to the successfully matched position state as the candidate key frame to be identified. Since the preset action is an action that causes the anti-counterfeiting point of the standard certificate to change, and the type of the standard certificate is the same as that of the certificate to be identified, the identification result of the certificate to be identified can be determined based on the candidate key frame. In the embodiment of the present application, when authenticating the certificate, only the video frame sequence needs to be obtained, which is independent of the certificate entity, greatly expanding the application scope of certificate authenticity identification.

[0237] For the specific definition of the video data processing device, please refer to the definition of the video data processing method above and will not be repeated here. Each module in the above-mentioned video data processing device can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each of the above modules.

[0238] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 9 As shown. The computer device includes a processor, memory, a communication interface, a display screen, and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal via wired or wireless communication. The wireless communication method can be achieved through Wi-Fi, a carrier network, NFC (near-field communication), or other technologies. When executed by the processor, the computer program implements a video data processing method. The display screen of the computer device can be a liquid crystal display or an electronic ink display. The input device of the computer device can be a touch layer covering the display screen, or keys, a trackball, or a touchpad provided on the computer device housing, or an external keyboard, touchpad, or mouse.

[0239] Those skilled in the art will understand that Figure 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0240] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0241] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0242] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of each of the above-described method embodiments.

[0243] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the embodiments provided herein may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0244] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0245] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A video data processing method, characterized in that: The method comprises: Obtain a video frame sequence containing the document to be authenticated; Determining the position of the document to be authenticated in the video frame sequence; determining a plurality of position states of the certificate to be authenticated in the video frame sequence according to the position of the certificate to be authenticated in the video frame sequence; Matching each of the position states with a target position state corresponding to a preset action, and determining the video frame corresponding to the successfully matched position state as a candidate key frame to be identified; the preset action is an action that causes the anti-counterfeiting point of the standard document to change, and the standard document is of the same type as the document to be identified; An authentication result of the document to be authenticated is determined based on the candidate key frames.

2. The method according to claim 1, characterized in that Determining the position of the certificate to be authenticated in the video frame sequence includes: Determining an initial video frame from the video frame sequence according to a time sequence of the video frames and obtaining a position of the document to be authenticated in the initial video frame; According to the position of the document to be authenticated in the initial video frame, target tracking is performed on the video frames following the initial video frame in the video frame sequence to determine the position of the document to be authenticated in the video frames following the initial video frame.

3. The method according to claim 2, characterized in that Determining an initial video frame from the video frame sequence and obtaining a position of the document to be authenticated in the initial video frame according to the time sequence of the video frames in the video frame sequence includes: Obtaining a trained target detection model; the target detection model includes a convolutional layer and a fully connected layer; Determining a current video frame from the video frame sequence according to a time sequence of the video frames in the video frame sequence; Inputting the current video frame into the convolution layer, performing feature extraction on the current video frame through the convolution layer, and obtaining a feature map corresponding to the current video frame; Inputting the feature map into the fully connected layer, performing document category prediction and position regression through the fully connected layer, and obtaining a document category prediction result and position corresponding to the current video frame; When the document category prediction result corresponding to the current video frame is used to represent the target category, the current video frame is determined as the initial video frame; When the document category prediction result corresponding to the current video frame is used to represent a non-target category, the current video frame is re-determined from the video frames after the current video frame in the video frame sequence and the step of inputting the current video frame into the convolution layer is returned.

4. The method according to claim 1, wherein Determining the position of the certificate to be authenticated in the video frame sequence includes: Determining an initial video frame from the video frame sequence according to a time sequence of the video frames and obtaining a position of the document to be authenticated in the initial video frame; Cutting out an image of the area where the document to be authenticated is located from the initial video frame according to the position corresponding to the initial video frame, and using the image as a template image; determining a search graph from video frames subsequent to the initial video frame; Inputting the template image and the search image into a trained target tracking model, performing target tracking processing on the search image based on the template image by the target tracking model, and obtaining a tracking result corresponding to the search image; The position of the document to be authenticated in the search graph is determined according to the tracking result.

5. The method according to claim 4, characterized in that The trained target tracking model includes a first convolution layer, a correlation layer, a second convolution layer, a deconvolution layer, and a refinement layer; inputting the template image and the search image into the trained target tracking model, and performing target tracking processing on the search image based on the template image by the target tracking model includes: Inputting the template image and the search image into the first convolutional layer respectively, performing feature extraction on the template image and the search image respectively through the first convolutional layer to obtain a template feature image corresponding to the template image and a search feature image corresponding to the search image; Inputting the template feature map and the search feature map into the correlation layer, cross-correlating the template feature map and the search feature map through the correlation layer to obtain a correlation feature map; the correlation feature map includes a plurality of candidate window responses; Inputting the relevant feature map into the second convolutional layer to obtain a score feature map and a mask feature map respectively; the mask feature map includes the mask feature submap corresponding to each candidate window response, and the score feature map includes the scores corresponding to each mask feature submap; Determining a candidate mask feature subgraph from the mask feature graph according to the scores corresponding to each mask feature subgraph, inputting the candidate mask feature subgraph into a deconvolution layer, and performing deconvolution processing on the candidate mask feature subgraph through the deconvolution layer to obtain a target mask feature subgraph; The target mask feature sub-image is input into the refinement layer, and the target mask feature sub-image is refined by the refinement layer to obtain a mask image corresponding to the search image.

6. The method according to claim 5, characterized in that The first convolution layer includes multiple convolution layers with successively smaller scales; inputting the template image and the search image into the first convolution layer, and performing feature extraction on the template image and the search image respectively through the first convolution layer includes: Performing multiple feature extractions on the search graph through the first convolutional layer, and saving multiple intermediate features obtained during the feature extraction process; The refinement layer includes multiple convolutional layers with successively larger scales; the target mask feature subgraph is input into the refinement layer, and the target mask feature subgraph is refined by the refinement layer to obtain a mask graph corresponding to the search graph, including: The target mask feature sub-image is upsampled multiple times through the refinement layer. Each time the image is upsampled, the output feature of the previous convolution layer is fused with the corresponding intermediate feature to obtain the current input feature. The current input feature is input into the current convolution layer and deconvolution is performed through the current convolution layer.

7. The method according to claim 1, characterized in that Determining, based on the position of the document to be authenticated in the video frame sequence, a plurality of position states of the document to be authenticated in the video frame sequence includes: filtering video frames in the video frame sequence that do not meet a preset condition according to the position of the certificate to be authenticated in the video frame sequence; Inputting the position of the document to be authenticated in the filtered video frame sequence into the trained position state prediction network; The position state of the document to be authenticated in the video frame sequence is determined according to the output result of the position state prediction network.

8. The method according to claim 1, characterized in that The candidate key frames include multiple frames; and determining the authentication result of the document to be authenticated based on the candidate key frames includes: Obtaining a quality score for each candidate key frame; the quality score is used to represent the clarity of the candidate key frame; Determine a target key frame according to the quality scores of each candidate key frame; An authentication result of the document to be authenticated is determined according to the target key frame.

9. The method according to claim 8, characterized in that Obtaining the quality score of each candidate key frame includes: Obtaining the horizontal gradient and vertical gradient of each color channel corresponding to the candidate key frame; Obtaining the gradient value of each color channel corresponding to the candidate key frame according to the horizontal gradient and the vertical gradient of each color channel corresponding to the candidate key frame; The gradient values ​​of each color channel are accumulated to obtain the quality score of the candidate key frame.

10. The method according to claim 8, characterized in that The preset actions include multiple ones, each of which corresponds to a different candidate key frame; Determining the target key frame according to the quality scores of the candidate key frames includes: The candidate key frames corresponding to the respective preset actions are respectively formed into candidate key frame sets corresponding to the respective preset actions; Determining target key frames of preset actions corresponding to respective candidate key frame sets according to quality scores of the candidate key frame sets in each candidate key frame set; Determining the authentication result of the document to be authenticated according to the target key frame includes: Perform channel splicing processing on each target key frame to obtain a spliced ​​key frame; The spliced ​​key frame is input into the trained identification result classification network to obtain the identification result of the document to be identified.

11. The method according to any one of claims 1 to 10, characterized in that Each of the position states includes direction information and angle information; the target position state includes target direction information and target angle information; and matching each of the position states with the target position state corresponding to the preset action, and determining the video frame corresponding to the successfully matched position state as the candidate key frame to be identified includes: Matching the direction information corresponding to each position state with the target direction information, and matching the angle information corresponding to each position state with the target angle information; A position state in which the direction information is consistent with the target direction information and the angle information is consistent with the target angle information is determined as a position state in which the matching is successful.

12. The method according to claim 11, characterized in that The angle information is an angle interval; the target angle information is a target angle interval; and determining the position state where the direction information is consistent with the target direction information and the angle information is consistent with the target angle information as a position state where the matching is successful includes: When the angle interval corresponding to any one position state is included in the target angle interval corresponding to the target position state, it is determined that the angle information corresponding to the position state is consistent with the target angle information.

13. A video data processing device, characterized in that: The device comprises: A video frame acquisition module, used to acquire a video frame sequence containing the document to be authenticated; A position determination module, configured to determine the position of the document to be authenticated in the video frame sequence; a position state determining module, configured to determine a plurality of position states of the document to be authenticated in the video frame sequence according to the position of the document to be authenticated in the video frame sequence; a matching module for matching each of the position states with a target position state corresponding to a preset action, and determining a video frame corresponding to a successfully matched position state as a candidate key frame to be identified; the preset action is an action that causes an anti-counterfeiting point of a standard document to change, and the standard document and the document to be identified are of the same type; The identification result determination module is used to determine the identification result of the document to be identified based on the candidate key frames.

14. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 12 are implemented.

15. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images