Certificate authenticity verification method and system, computer device and storage medium

Abstract

The application relates to a certificate authenticity verification method and system, computer equipment and a storage medium. The certificate authenticity verification method comprises the following steps: video acquisition of a certificate at different preset angles to obtain preprocessed video frames, and the preprocessed video frames are used as inputs of an end-to-end learning algorithm; two-dimensional feature data is extracted from the preprocessed video frames by a basic network, and the two-dimensional feature data is fused into corresponding three-dimensional feature data; three-dimensional feature data of a plurality of anti-counterfeiting regions is obtained from the three-dimensional feature data by using a RoIAlign tool to establish a multi-task learning model of the end-to-end learning algorithm; and a verification result of the certificate is output by the multi-task learning model of the end-to-end learning algorithm; wherein one learning task is established by using the three-dimensional feature data of each anti-counterfeiting region, so that the accuracy and efficiency of certificate authenticity verification are improved.

Description

Technical Field

[0001] This application relates to the field of document verification, and in particular to a method, system, computer device, and storage medium for verifying the authenticity of documents. Background Technology

[0002] In modern e-commerce, automated processes have become increasingly important due to their advantages in improving efficiency and reducing costs, such as EKYC (Electronic Know Your Customer) in financial systems. One key scenario involves users uploading photos of their ID documents; the system automatically verifies the authenticity of the ID and extracts relevant information to determine whether to proceed to the next step in the automated process.

[0003] Forgery of identity documents is not uncommon in reality, generally involving photocopying, counterfeiting, and photographing (i.e., using electronic devices to photograph the document). On the other hand, most identity documents have anti-counterfeiting features, which can be broadly categorized into static and dynamic features: static features refer to fixed and unchanging characteristics, such as special patterns, colors, etc.; dynamic features refer to features that change color or form under different angles or lighting conditions, such as color-changing ink and gradient text. In practical applications, dynamic features have stronger anti-counterfeiting capabilities due to their difficulty in forgery, and are therefore the preferred consideration for model algorithms.

[0004] Traditional machine learning methods typically employ handcrafted features, which are then fed as input into machine learning models such as logistic regression, support vector machines (SVM), random forests, and gradient boosting trees to obtain the final result. However, this pipeline approach has drawbacks: the design, learning, and generation of features are too artificial, requiring numerous trials and adjustments; moreover, feature learning and model training are disconnected, and the optimal generated features do not always correspond to the optimal machine learning model, making it difficult to obtain the optimal model. This results in low efficiency and accuracy in verifying the authenticity of documents. Summary of the Invention

[0005] Based on this, this application provides a method, system, computer device, and storage medium for verifying the authenticity of documents, so as to improve the efficiency and accuracy of document verification.

[0006] On the one hand, a method for verifying the authenticity of an identification document is provided, the method comprising:

[0007] Videos of the documents are captured from different preset angles to obtain pre-processed video frames, which are then used as input to an end-to-end learning algorithm.

[0008] Two-dimensional feature data is extracted from the preprocessed video frames through a basic network, and the two-dimensional feature data is fused into corresponding three-dimensional feature data.

[0009] The RoIAlign tool is used to obtain the three-dimensional feature data of multiple anti-counterfeiting areas from the three-dimensional feature data in order to establish a multi-task learning model of the end-to-end learning algorithm.

[0010] The multi-task learning model of the end-to-end learning algorithm outputs the verification result of the document;

[0011] One of the three-dimensional feature data of the anti-counterfeiting area is used to establish a learning task.

[0012] In one embodiment, the step of capturing video of the document at different preset angles to obtain preprocessed video frames, and using the preprocessed video frames as input to the end-to-end learning algorithm, includes:

[0013] Receive a video of a document flipped over, taken by the user according to the instructions, and obtain video frames from the video of the document flipped over;

[0014] Identify the coordinates of the four corners of the document in the video frame, and obtain video frames of the document at different flip angles based on the coordinates of the four corners;

[0015] The video frames of the document at different flip angles are detected to obtain video frames that meet preset conditions at different flip angles.

[0016] White balance processing is performed on video frames that meet preset conditions at different flip angles to obtain multiple pre-processed video frames.

[0017] Multiple preprocessed video frames are used as input to the end-to-end learning algorithm.

[0018] In one embodiment, the step of obtaining two-dimensional feature data from the preprocessed video frames via a basic network and fusing the two-dimensional feature data into corresponding three-dimensional feature data includes:

[0019] The preprocessed video frames are subjected to feature extraction by a basic network in a preset angular order to obtain two-dimensional feature data of the preprocessed video frames in a preset angular order.

[0020] The two-dimensional feature data arranged in a preset angular order are stacked to fuse them into the three-dimensional feature data;

[0021] Perform a three-dimensional convolution operation on the three-dimensional feature data to obtain three-dimensional dynamic feature data.

[0022] In one embodiment, obtaining the three-dimensional feature data of multiple anti-counterfeiting regions from the three-dimensional feature data using the RoIAlign tool includes:

[0023] Based on the feature information of the anti-counterfeiting area, the RoIAlign tool is used to extract the three-dimensional dynamic feature data of different anti-counterfeiting areas from the three-dimensional dynamic feature data.

[0024] In one embodiment, establishing the multi-task learning model of the end-to-end learning algorithm includes:

[0025] Different learning tasks are established for the three-dimensional dynamic feature data of different anti-counterfeiting areas to build a multi-task learning model of the end-to-end learning algorithm.

[0026] The learning task also includes a learning task for establishing the three-dimensional dynamic feature data of the entire document.

[0027] In one embodiment, the method further includes:

[0028] The loss function of the end-to-end learning algorithm is the sum of the loss functions of different learning tasks;

[0029] The loss function for different learning tasks is configured with corresponding weights.

[0030] In one embodiment, the multi-task learning model outputting the verification result of the document through the end-to-end learning algorithm includes:

[0031] The multi-task learning model of the end-to-end learning algorithm outputs the verification result of the document;

[0032] The verification results include the verification results of the authenticity of the document and the binary classification results of each learning task.

[0033] On the other hand, a document authenticity verification system is provided, the document authenticity verification system comprising:

[0034] The algorithm input module captures video of the document from different preset angles to obtain pre-processed video frames, and uses the pre-processed video frames as input to the end-to-end learning algorithm.

[0035] The algorithm model module obtains two-dimensional feature data from the preprocessed video frames through a basic network, and fuses the two-dimensional feature data into corresponding three-dimensional feature data; it obtains three-dimensional feature data of multiple anti-counterfeiting areas from the three-dimensional feature data through the RoIAlign tool to establish a multi-task learning model of the end-to-end learning algorithm, wherein a learning task is established through the three-dimensional feature data of each anti-counterfeiting area.

[0036] The algorithm output module outputs the verification result of the document through the multi-task learning model of the end-to-end learning algorithm.

[0037] In another aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, performs the following steps:

[0038] Videos of the documents are captured from different preset angles to obtain pre-processed video frames, which are then used as input to an end-to-end learning algorithm.

[0039] Two-dimensional feature data is extracted from the preprocessed video frames through a basic network, and the two-dimensional feature data is fused into corresponding three-dimensional feature data.

[0040] The RoIAlign tool is used to obtain the three-dimensional feature data of multiple anti-counterfeiting areas from the three-dimensional feature data in order to establish a multi-task learning model of the end-to-end learning algorithm.

[0041] The verification result of the document is output through the multi-task learning model of the end-to-end learning algorithm.

[0042] In another aspect, a computer-readable storage medium is provided, the computer-readable storage medium storing a program that, when executed by a processor, causes the processor to perform the following steps:

[0043] Videos of the documents are captured from different preset angles to obtain pre-processed video frames, which are then used as input to an end-to-end learning algorithm.

[0044] Two-dimensional feature data is extracted from the preprocessed video frames through a basic network, and the two-dimensional feature data is fused into corresponding three-dimensional feature data.

[0045] The RoIAlign tool is used to obtain the three-dimensional feature data of multiple anti-counterfeiting areas from the three-dimensional feature data in order to establish a multi-task learning model of the end-to-end learning algorithm.

[0046] The verification result of the document is output through the multi-task learning model of the end-to-end learning algorithm.

[0047] The technical solution described in this application has the following advantages over the prior art:

[0048] (1) An end-to-end learning algorithm is adopted, which only focuses on the input of raw data to the output of task results. That is, the pre-processed video frames at different angles are used as the input of the end-to-end learning algorithm, and the output is the verification result of the authenticity of the certificate. This achieves global optimization and improves the accuracy of the authenticity verification. In addition, the modeling and training of the end-to-end method is simpler and faster, and the input is the original image and the output is the final result.

[0049] (2) In the end-to-end learning algorithm, the RoIAlign tool is combined to extract the three-dimensional feature data of the anti-counterfeiting area. The bilinear interpolation method is used to calculate the feature map (feature data) of each RoI, which can fully retain the key area information and improve the accuracy of the feature data.

[0050] (3) The end-to-end learning algorithm also establishes a multi-task learning mechanism for multiple anti-counterfeiting areas of the certificate. In addition to obtaining the final authenticity of the certificate, it can also obtain the feature scores of each anti-counterfeiting area, which is beneficial for subsequent processing or manual analysis. Moreover, the multi-task learning method makes the algorithm model more robust and generalizable. In other words, the model that performs well on the training dataset also performs well on the test dataset and in actual production, effectively avoiding the overfitting problem.

[0051] (4) On the same production dataset, the false certificate rejection rate of traditional machine learning methods is about 1%, and the true certificate pass rate is about 75%; while the solution of this application can reject almost all false certificates, and the true certificate pass rate is over 90%, which has a very significant performance improvement.

[0052] The aforementioned methods, systems, computer equipment, and storage media for verifying the authenticity of documents effectively improve the accuracy of document verification. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a flowchart of the first method of the document authenticity verification method provided in the embodiments of this application;

[0055] Figure 2 This is a flowchart of the second method of the document authenticity verification method provided in the embodiments of this application;

[0056] Figure 3 This is a flowchart of the end-to-end learning algorithm for the document authenticity verification method provided in the embodiments of this application;

[0057] Figure 4 This is a system structure diagram of the document authenticity verification system provided in the embodiments of this application;

[0058] Figure 5 This is a device structure diagram of the computer device provided in the embodiments of this application;

[0059] Figure 6 This is an example image of a certificate provided in an embodiment of this application. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0061] like Figure 6 The image shown is an example of an ID card. The ID card's anti-counterfeiting features include static and dynamic features. Dynamic features include: the triangular color-changing ink in selected area 1 (changing color from orange to green at different angles); the character gradient feature in selected area 2 (the character gradually changes from H to K at different angles); and the small face gradient feature in selected area 3 (the image clarity of the small face changes at different angles). Static features include the material of the ID card, its pattern, and its color gradient. This application aims to verify the authenticity of ID cards with dynamic anti-counterfeiting features.

[0062] Example 1:

[0063] Reference Figure 1 and Figure 3 As shown, Figure 1 A flowchart of the first method for verifying the authenticity of documents provided in this application embodiment; Figure 3 The flowchart shows the end-to-end learning algorithm of the document authenticity verification method provided in the embodiments of this application.

[0064] The method includes the following steps:

[0065] S101, capture video of the document from different preset angles to obtain pre-processed video frames, and use the pre-processed video frames as input to the end-to-end learning algorithm;

[0066] Specifically, when verifying the authenticity of a document, the first step is to collect information from the document, primarily through images and / or video capture of the document's surface. During video capture, the user rotates the document at certain angles under the camera's guidance to obtain video footage from different perspectives. This video footage is then preprocessed to obtain preprocessed video frames, which are used as input to the end-to-end learning algorithm. The end-to-end learning algorithm does not involve manual sub-problem division; instead, it allows the deep learning model to directly learn the mapping from raw data to the desired output. By reducing manual preprocessing and subsequent processing, the algorithm aims to provide the model with more flexibility to automatically adjust based on data, increasing overall model fit and ultimately achieving the globally optimal solution.

[0067] S102, extract two-dimensional feature data from the preprocessed video frame through the basic network, and fuse the two-dimensional feature data into corresponding three-dimensional feature data;

[0068] Specifically, after acquiring the preprocessed video frames, feature extraction is performed on the preprocessed video frames using basic networks such as EfficientNet and ResNet. Two-dimensional feature data is extracted from the preprocessed video frames. Two-dimensional feature data refers to the two-dimensional planar feature data collected from the document, that is, the two-dimensional feature map. After acquiring the two-dimensional feature data, since the two-dimensional feature data includes the two-dimensional feature data of the document under different flip angles, these two-dimensional feature data under different flip angles are fused to form three-dimensional feature data. The three-dimensional feature data reflects the interrelationship of the two-dimensional feature data under different flip angles. For example, the three-dimensional feature data of the triangular color-changing ink in the selected area 1 can reflect the change relationship between different colored inks, so as to further determine the authenticity of the document through the three-dimensional feature data.

[0069] S103, the three-dimensional feature data of multiple anti-counterfeiting areas are obtained from the three-dimensional feature data using the RoIAlign tool, so as to establish a multi-task learning model of the end-to-end learning algorithm;

[0070] Specifically, the 3D feature data includes the data of the entire document. Verifying the authenticity of the document involves verifying the authenticity of the areas on the document that possess dynamic anti-counterfeiting features. Therefore, after obtaining the 3D feature data, the 3D feature data of the anti-counterfeiting areas needs to be extracted from the document's 3D feature data using the RoIAlign (region of interest align) tool, such as... Figure 6The document shown includes three dynamic anti-counterfeiting areas: the triangular color-changing ink in selected area 1, the character gradient feature in selected area 2, and the small face gradient feature in selected area 3. After obtaining the 3D feature data of the three anti-counterfeiting areas, three learning tasks are established for the end-to-end learning algorithm, forming a multi-task learning model for the end-to-end learning algorithm. Specifically, the 3D feature data of one anti-counterfeiting area is used to establish one learning task. The RoIAlign tool is used to extract the anti-counterfeiting area features. Because multiple samplings in the convolutional network convert the RoI (region of interest) coordinates into fractions, direct rounding would cause information loss and easily lead to a decrease in model performance. Therefore, the RoIAlign tool is used for feature data acquisition. The RoIAlign tool uses bilinear interpolation to calculate the feature map (feature data) of each RoI, which can fully retain key area information and improve the accuracy of document authenticity verification.

[0071] S104, the verification result of the document is output through the multi-task learning model of the end-to-end learning algorithm;

[0072] Specifically, after the multi-task learning model calculates the three-dimensional feature data of multiple anti-counterfeiting areas, the end-to-end learning algorithm can output the result of document authenticity verification through the multi-task learning model.

[0073] This application implements document authenticity verification through an end-to-end learning algorithm, focusing solely on the input of raw data and the output of the task result. Specifically, it uses preprocessed video frames from different angles as input to the end-to-end learning algorithm, outputting the verification result of document authenticity to achieve global optimization and improve the accuracy of document authenticity verification. Furthermore, the end-to-end method is simpler and faster to model and train, inputting the original image and outputting the final result. The end-to-end learning algorithm incorporates the RoIAlign tool, extracting the 3D feature data of the anti-counterfeiting area and using bilinear interpolation to calculate the feature map (feature data) of each RoI, which fully preserves key area information and improves the accuracy of feature data. In addition, the end-to-end learning algorithm establishes a multi-task learning mechanism for multiple anti-counterfeiting areas of the document. Besides obtaining the final document authenticity, it also obtains the feature scores of each anti-counterfeiting area, which is beneficial for subsequent processing or manual analysis. The multi-task learning method makes the algorithm model more robust and generalizable; that is, a model that performs well on the training dataset also performs well on the test dataset and in actual production, effectively avoiding overfitting. On the same production dataset, traditional machine learning methods achieve a fake document rejection rate of around 1% and a genuine document pass rate of around 75%. In contrast, the proposed solution can reject almost all fake documents while achieving a genuine document pass rate exceeding 90%, demonstrating a significant performance improvement. Furthermore, the proposed solution is not limited to specific ID documents and can be applied to any ID document scenario with dynamic anti-counterfeiting features.

[0074] In one embodiment, the step of capturing video of the document at different preset angles to obtain preprocessed video frames, and using the preprocessed video frames as input to the end-to-end learning algorithm, includes:

[0075] Receive a video of a document flipped over, taken by the user according to the instructions, and obtain video frames from the video of the document flipped over;

[0076] Specifically, the user rotates the ID document at certain angles under the camera capture device as instructed, to obtain video of the ID document from different angles, i.e., ID document flipping video. The end-to-end learning algorithm receives the ID document flipping video taken by the user according to the instructions, and then extracts each frame of video from the ID document flipping video, i.e., video frames.

[0077] Identify the coordinates of the four corners of the document in the video frame, and obtain video frames of the document at different flip angles based on the coordinates of the four corners;

[0078] Specifically, after acquiring the video frame, it can be identified to determine the coordinates of the four corners of the document. By calculating the coordinates of the four corners of the document, the flip angle of the document in the video frame can be obtained. When determining the authenticity of the document, images of the document at different angles are needed. Therefore, after acquiring the video frame, the corner coordinates of the video frame are identified and calculated to obtain video frames at multiple flip angles. Feature data is then extracted from these video frames at different flip angles to further verify the authenticity of the document.

[0079] The video frames of the document at different flip angles are detected to obtain video frames that meet preset conditions at different flip angles.

[0080] Specifically, after acquiring video frames at different flip angles, the video frames at different flip angles are inspected to determine whether they meet preset conditions, i.e., whether they meet quality requirements such as clarity and completeness. Multiple video frames at different flip angles are sampled to select those that meet the preset conditions; for example, 10 frames are sampled, ensuring they have various spatial orientations and are arranged according to their flip angle.

[0081] White balance processing is performed on video frames that meet preset conditions at different flip angles to obtain multiple pre-processed video frames.

[0082] Specifically, after acquiring video frames that meet preset conditions at different flip angles, preprocessing is required. In actual document photography, documents may be captured in various environmental scenarios, such as different lighting and color tones, which can affect subsequent data extraction. In AI algorithms, improving data quality is a faster and easier method than adjusting the network architecture because data preprocessing limits the range of input data, thus simplifying the algorithm problem. Therefore, white balance processing is performed on the images (video frames) to eliminate the influence of lighting conditions on the algorithm model, resulting in preprocessed video frames.

[0083] Multiple preprocessed video frames are used as input to the end-to-end learning algorithm.

[0084] Specifically, after obtaining the preprocessed video frames, the obtained preprocessed video frames are used as input to the end-to-end learning algorithm so that the model output of the end-to-end learning algorithm can verify the results.

[0085] In one embodiment, the step of obtaining two-dimensional feature data from the preprocessed video frames via a basic network and fusing the two-dimensional feature data into corresponding three-dimensional feature data includes:

[0086] The preprocessed video frames are subjected to feature extraction by a basic network in a preset angular order to obtain two-dimensional feature data of the preprocessed video frames in a preset angular order.

[0087] Specifically, the preprocessed video frames include images at different flip angles. After the preprocessed video frames are acquired, the multiple preprocessed video frames are sorted according to their flip angles. Then, the sorted preprocessed video frames are processed by a basic network to extract features, thereby obtaining two-dimensional feature data of the preprocessed video frames in a preset angle order.

[0088] The two-dimensional feature data arranged in a preset angular order are stacked to fuse them into the three-dimensional feature data;

[0089] Specifically, after obtaining the two-dimensional feature data of the preprocessed video frames in a preset angular order, it is necessary to convert the two-dimensional feature data into three-dimensional feature data. That is, the two-dimensional feature data in the preset angular order is stacked to fuse into three-dimensional feature data. Here, the two-dimensional feature data includes two-dimensional feature data with different flip angles, so the three-dimensional feature data reflects the relationship between the two-dimensional feature data with different flip angles.

[0090] Perform a three-dimensional convolution operation on the three-dimensional feature data to obtain three-dimensional dynamic feature data.

[0091] Specifically, after acquiring the 3D feature data, it is necessary to effectively reflect the dynamic relationships between 2D data from different angles using this data. Therefore, the acquired 3D feature data is subjected to 3D convolution operations to obtain 3D dynamic feature data. This 3D dynamic feature data can effectively reflect the changes and relationships between adjacent video frames. In fact, adjacent images (video frames) can be viewed as sequences, and naturally, we can also consider using recurrent neural networks (RNNs) or the recently popular attention mechanism to learn these features.

[0092] In one embodiment, obtaining the three-dimensional feature data of multiple anti-counterfeiting regions from the three-dimensional feature data using the RoIAlign tool includes:

[0093] Based on the feature information of the anti-counterfeiting area, the RoIAlign tool is used to extract the three-dimensional dynamic feature data of different anti-counterfeiting areas from the three-dimensional dynamic feature data.

[0094] Specifically, to effectively reflect the changes and relationships between adjacent video frames, it is necessary to use three-dimensional dynamic feature data. Therefore, the RoIAlign tool is needed to extract the three-dimensional dynamic feature data of the anti-counterfeiting area from the three-dimensional dynamic feature data of the document. For example... Figure 6 The document shown requires three-dimensional dynamic feature data of three anti-counterfeiting areas.

[0095] In one embodiment, establishing the multi-task learning model of the end-to-end learning algorithm includes:

[0096] Different learning tasks are established for the three-dimensional dynamic feature data of different anti-counterfeiting areas to build a multi-task learning model of the end-to-end learning algorithm.

[0097] The learning task also includes a learning task for establishing the three-dimensional dynamic feature data of the entire document.

[0098] Specifically, after acquiring the 3D dynamic feature data of the three anti-counterfeiting areas, three learning tasks are established based on the 3D dynamic feature data of the three anti-counterfeiting areas to form a multi-task learning model of the end-to-end learning algorithm. Each of the 3D feature data of the anti-counterfeiting area establishes one learning task. Furthermore, the learning tasks also include a learning task for establishing the 3D dynamic feature data of the entire document, for example... Figure 6 The certificate shown can be used to create four learning tasks, such as Figure 3 The four learning tasks shown include learning the triangular color-changing ink area, learning the character gradient area, learning the small face gradient area, and learning the entire document.

[0099] In one embodiment, the method further includes:

[0100] The loss function of the end-to-end learning algorithm is the sum of the loss functions of different learning tasks;

[0101] The loss function for different learning tasks is configured with corresponding weights.

[0102] Specifically, each learning task corresponds to a loss function, and four learning tasks include four loss functions. The loss function of the entire end-to-end learning algorithm is represented by the sum of the four loss functions. In addition, the loss function of each learning task can be set with corresponding weights to adjust the influence of the four loss functions.

[0103] In one implementation, the multi-task learning model outputting the verification result of the document through the end-to-end learning algorithm includes:

[0104] The multi-task learning model of the end-to-end learning algorithm outputs the verification result of the document;

[0105] The verification results include the verification results of the authenticity of the document and the binary classification results of each learning task.

[0106] Specifically, a multi-task learning model using an end-to-end learning algorithm can output verification results for the document. These results include not only the overall verification of the document but also the binary classification results for each anti-counterfeiting area, for later analysis. The binary classification result, indicating whether the document is genuine or counterfeit, is a number between 0 and 1 for the model's prediction; a higher number indicates a greater probability of authenticity, hence the term "score." For example, a score of 0.7 or higher indicates a genuine document and verification passes; a score below 0.4 indicates a counterfeit document and verification fails; scores between 0.4 and 0.7 are submitted for manual review, and the final verification result is determined by the manual review process.

[0107] Example 2:

[0108] Reference Figure 2 and Figure 3 As shown, Figure 2 This is a flowchart of the second method for verifying the authenticity of documents provided in an embodiment of this application. Figure 2 In the method shown, with Figure 1 and Figure 3 For content that is the same or similar to the method shown, please refer to... Figure 1 and Figure 3 The method description will not be repeated here.

[0109] S201, Receive the document flipping video taken by the user according to the instruction information, and obtain video frames from the document flipping video;

[0110] Users follow instructions to rotate their identification documents at certain angles using a camera capture device to obtain videos of the documents from different angles, i.e., document flip videos. An end-to-end learning algorithm receives these document flip videos captured by the user according to the instructions, and then extracts each frame of the video from them.

[0111] S202, identify the four corner coordinates of the document in the video frame, so as to obtain the video frames of the document at different flip angles based on the four corner coordinates;

[0112] After acquiring the video frames, they can be identified to determine the coordinates of the four corners of the document. By calculating these coordinates, the flip angle of the document in the video frame can be obtained. Since determining the authenticity of the document requires images of it from different angles, the corner coordinates of the video frames are identified and calculated to obtain video frames at multiple flip angles. Feature data is then extracted from these video frames at different flip angles to further verify the authenticity of the document.

[0113] S203, detect the video frames of the document at different flip angles to obtain video frames that meet the preset conditions at different flip angles;

[0114] After acquiring video frames at different flip angles, the video frames at different flip angles are detected to determine whether they meet preset conditions, that is, whether they meet quality requirements, such as clarity and completeness.

[0115] S204, perform white balance processing on video frames that meet preset conditions under different flip angles to obtain multiple pre-processed video frames;

[0116] After obtaining video frames that meet the preset conditions at different flip angles, the video frames need to be preprocessed. Since the documents may be in different environmental scenarios during actual shooting, such as different lighting and color tones, these environments will affect the subsequent data extraction. Therefore, white balance processing is performed on the images (video frames) to eliminate the influence of lighting environment on the algorithm model in order to obtain preprocessed video frames.

[0117] S205, the multiple preprocessed video frames are used as input to the end-to-end learning algorithm;

[0118] After obtaining the preprocessed video frames, the obtained preprocessed video frames are used as input to the end-to-end learning algorithm so that the model output of the end-to-end learning algorithm can verify the results.

[0119] S206, the preprocessed video frames are subjected to feature extraction by a basic network in a preset angle order to obtain two-dimensional feature data of the preprocessed video frames in a preset angle order;

[0120] The preprocessed video frames include images at different flip angles. After the preprocessed video frames are acquired, they are sorted according to their flip angles. Then, the sorted preprocessed video frames are processed by a basic network to extract features, thereby obtaining two-dimensional feature data of the preprocessed video frames in a preset angle order.

[0121] S207, The two-dimensional feature data arranged in a preset angle order are stacked to fuse into the three-dimensional feature data;

[0122] After obtaining the two-dimensional feature data of the preprocessed video frames in a preset angle order, it is necessary to convert the two-dimensional feature data into three-dimensional feature data. That is, the two-dimensional feature data in a preset angle order is stacked to fuse into three-dimensional feature data.

[0123] S208, Perform a three-dimensional convolution operation on the three-dimensional feature data to obtain three-dimensional dynamic feature data;

[0124] After obtaining the 3D feature data, it is necessary to effectively reflect the dynamic relationship between 2D data from different angles. Therefore, the obtained 3D feature data is subjected to 3D convolution operation to obtain 3D dynamic feature data. The 3D dynamic feature data can effectively reflect the changes and relationships between adjacent video frames.

[0125] S209, according to the feature information of the anti-counterfeiting area, the three-dimensional dynamic feature data of different anti-counterfeiting areas is extracted from the three-dimensional dynamic feature data using the RoIAlign tool;

[0126] To effectively reflect the changes and relationships between adjacent video frames, it is necessary to use three-dimensional dynamic feature data. Therefore, the RoIAlign tool is needed to extract the three-dimensional dynamic feature data of the anti-counterfeiting area from the three-dimensional dynamic feature data of the document.

[0127] S210, different learning tasks are established for the three-dimensional dynamic feature data of different anti-counterfeiting areas to establish a multi-task learning model of the end-to-end learning algorithm;

[0128] After obtaining the 3D dynamic feature data of the three anti-counterfeiting areas, three learning tasks are established based on the 3D dynamic feature data of the three anti-counterfeiting areas to form a multi-task learning model of the end-to-end learning algorithm. Specifically, a learning task is established for the 3D feature data of each anti-counterfeiting area. Furthermore, the learning tasks also include a learning task for establishing the 3D dynamic feature data of the entire document, for example... Figure 6 The certificate shown can be used to create four learning tasks, such as Figure 3 The four learning tasks shown include learning the triangular color-changing ink area, learning the character gradient area, learning the small face gradient area, and learning the entire document.

[0129] S211, the verification result of the document is output through the multi-task learning model of the end-to-end learning algorithm.

[0130] The multi-task learning model using an end-to-end learning algorithm can output the verification results of the document. The verification results include not only the verification results of the entire document, but also the binary classification results of each anti-counterfeiting area, which can be used for subsequent result analysis.

[0131] It should be understood that, although Figures 1-2 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figures 1-2At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0132] Example 3:

[0133] Reference Figure 4 As shown, Figure 4 The system structure diagram of the document authenticity verification system provided in the embodiments of this application is shown.

[0134] The document authenticity verification system of this embodiment includes:

[0135] The algorithm input module captures video of the document from different preset angles to obtain pre-processed video frames, and uses the pre-processed video frames as input to the end-to-end learning algorithm.

[0136] When verifying the authenticity of an identification document, the first step is to collect information from the document, primarily images and / or video of the document's surface. During video capture, the user rotates the document at certain angles as instructed by the camera to obtain video footage from different perspectives. The algorithm input module then preprocesses these preprocessed video frames, which are used as input to the end-to-end learning algorithm.

[0137] The algorithm model module obtains two-dimensional feature data from the preprocessed video frames through a basic network, and fuses the two-dimensional feature data into corresponding three-dimensional feature data; it obtains three-dimensional feature data of multiple anti-counterfeiting areas from the three-dimensional feature data through the RoIAlign tool to establish a multi-task learning model of the end-to-end learning algorithm, wherein a learning task is established through the three-dimensional feature data of each anti-counterfeiting area.

[0138] After acquiring the preprocessed video frames, the algorithm model module extracts features from them, extracting two-dimensional feature data. This two-dimensional feature data refers to the two-dimensional planar feature data collected from the document, i.e., a two-dimensional feature map. After obtaining the two-dimensional feature data through the algorithm model module, since this data includes the document's two-dimensional feature data at different flip angles, these data are fused to form three-dimensional feature data. This three-dimensional feature data reflects the interrelationships between the two-dimensional feature data at different flip angles. The three-dimensional feature data includes the entire document's data. Verifying the authenticity of the document involves verifying the authenticity of the areas with dynamic anti-counterfeiting features. Therefore, after obtaining the three-dimensional feature data, the algorithm model module needs to extract the three-dimensional feature data of the anti-counterfeiting areas from the document's three-dimensional feature data.

[0139] The algorithm output module outputs the verification result of the document through the multi-task learning model of the end-to-end learning algorithm.

[0140] After the multi-task learning model calculates the 3D feature data of multiple anti-counterfeiting areas, the end-to-end learning algorithm can output the result of the document authenticity verification through the algorithm output module.

[0141] In one embodiment, the document authenticity verification system further includes:

[0142] The loss function module configures the loss function for the entire end-to-end learning algorithm;

[0143] Each learning task corresponds to a loss function, and four learning tasks include four loss functions. The loss function of the entire end-to-end learning algorithm is represented by the sum of the four loss functions. In addition, the loss function of each learning task can be set with corresponding weights to adjust the influence of the four loss functions. The loss function module is used to configure the loss functions.

[0144] Specific limitations regarding the document verification system can be found in the method limitations section above, and will not be repeated here. Each module in the aforementioned document verification system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0145] Example 4:

[0146] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. The feature is that when the processor executes the computer program, it implements the steps of a document authenticity verification method.

[0147] The computer device can be a terminal, and its internal structure diagram can be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for verifying the authenticity of documents. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0148] Those skilled in the art should understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0149] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0150] Videos of the documents are captured from different preset angles to obtain pre-processed video frames, which are then used as input to an end-to-end learning algorithm.

[0151] Two-dimensional feature data is extracted from the preprocessed video frames through a basic network, and the two-dimensional feature data is fused into corresponding three-dimensional feature data.

[0152] The RoIAlign tool is used to obtain the three-dimensional feature data of multiple anti-counterfeiting areas from the three-dimensional feature data in order to establish a multi-task learning model of the end-to-end learning algorithm.

[0153] The verification result of the document is output through the multi-task learning model of the end-to-end learning algorithm.

[0154] Example 5:

[0155] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it performs the following steps:

[0156] Videos of the documents are captured from different preset angles to obtain pre-processed video frames, which are then used as input to an end-to-end learning algorithm.

[0157] Two-dimensional feature data is extracted from the preprocessed video frames through a basic network, and the two-dimensional feature data is fused into corresponding three-dimensional feature data.

[0158] The RoIAlign tool is used to obtain the three-dimensional feature data of multiple anti-counterfeiting areas from the three-dimensional feature data in order to establish a multi-task learning model of the end-to-end learning algorithm.

[0159] The verification result of the document is output through the multi-task learning model of the end-to-end learning algorithm.

[0160] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0161] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0162] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for verifying the authenticity of a document, characterized in that, The document authenticity verification method includes: Videos of the documents are captured from different preset angles to obtain pre-processed video frames, which are then used as input to an end-to-end learning algorithm. Two-dimensional feature data is extracted from the preprocessed video frames through a basic network, and the two-dimensional feature data is fused into corresponding three-dimensional feature data. The RoIAlign tool is used to obtain three-dimensional feature data of multiple anti-counterfeiting areas from the three-dimensional feature data, so as to establish different learning tasks for the three-dimensional dynamic feature data of different anti-counterfeiting areas, in order to establish a multi-task learning model of the end-to-end learning algorithm. The learning tasks also include the learning task established for the three-dimensional dynamic feature data of the entire document. The multi-task learning model of the end-to-end learning algorithm outputs the verification result of the document, which includes the document authenticity verification result and the binary classification result of each learning task. The step of obtaining two-dimensional feature data from the preprocessed video frames through a basic network and fusing the two-dimensional feature data into corresponding three-dimensional feature data includes: The preprocessed video frames are subjected to feature extraction by a basic network in a preset angular order to obtain two-dimensional feature data of the preprocessed video frames in a preset angular order. The two-dimensional feature data arranged in a preset angular order are stacked to fuse them into the three-dimensional feature data; Perform a 3D convolution operation on the 3D feature data to obtain 3D dynamic feature data; The loss function of the end-to-end learning algorithm is the sum of the loss functions of different learning tasks; In this context, a learning task is established using the three-dimensional feature data of one of the anti-counterfeiting areas, and the loss function for different learning tasks is configured with corresponding weights.

2. The document authenticity verification method according to claim 1, characterized in that, The step of capturing video of the document at different preset angles to obtain pre-processed video frames, and using the pre-processed video frames as input to the end-to-end learning algorithm, includes: Receive a video of a document flipped over, taken by the user according to the instructions, and obtain video frames from the video of the document flipped over; Identify the coordinates of the four corners of the document in the video frame, and obtain video frames of the document at different flip angles based on the coordinates of the four corners; The video frames of the document at different flip angles are detected to obtain video frames that meet preset conditions at different flip angles. White balance processing is performed on video frames that meet preset conditions at different flip angles to obtain multiple pre-processed video frames. Multiple preprocessed video frames are used as input to the end-to-end learning algorithm.

3. The document authenticity verification method according to claim 1, characterized in that, The process of obtaining 3D feature data of multiple anti-counterfeiting regions from the 3D feature data using the RoIAlign tool includes: Based on the feature information of the anti-counterfeiting area, the RoIAlign tool is used to extract the three-dimensional dynamic feature data of different anti-counterfeiting areas from the three-dimensional dynamic feature data.

4. A document authenticity verification system that implements the document authenticity verification method as described in any one of claims 1-3, characterized in that, The document authenticity verification system includes: The algorithm input module captures video of the document from different preset angles to obtain pre-processed video frames, and uses the pre-processed video frames as input to the end-to-end learning algorithm. The algorithm model module obtains two-dimensional feature data from the preprocessed video frames through a basic network, and fuses the two-dimensional feature data into corresponding three-dimensional feature data; it obtains three-dimensional feature data of multiple anti-counterfeiting areas from the three-dimensional feature data through the RoIAlign tool to establish a multi-task learning model of the end-to-end learning algorithm, wherein a learning task is established through the three-dimensional feature data of each anti-counterfeiting area. The algorithm output module outputs the verification result of the document through the multi-task learning model of the end-to-end learning algorithm.

5. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program that, when executed by a processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 3.

Images