Certificate identification method and device based on artificial intelligence, equipment and medium
By using an AI-based document recognition method, which utilizes feature extraction models and similarity calculations to identify documents, the problem of low efficiency and low accuracy in online document verification has been solved, achieving fast and accurate document recognition and management of forged information.
Patent Information
- Application Number
- CN202511060468.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-30
- Publication Date
- 2025-11-14
AI Technical Summary
Existing technologies for online document verification are inefficient, human verification relies on work experience, and OCR recognition has low accuracy, making it difficult to effectively identify counterfeit documents such as those with forged codes and alterations.
An AI-based document recognition method is adopted. The feature extraction model extracts the features of the document image, calculates the similarity between the document features and the vector library of counterfeit documents, determines whether the maximum similarity exceeds the threshold, identifies counterfeit documents, and adds the counterfeit document information to the vector library.
It enables rapid identification of newly discovered forged document information, takes effect in real time, simplifies operations, saves time and money, and improves the accuracy and efficiency of document recognition.
Smart Images
Figure CN120953774A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and in particular to an artificial intelligence-based document recognition method, apparatus, device, and medium. Background Technology
[0002] With the widespread adoption of the internet, online business transactions and other services have gradually become important business processing methods in industries such as finance and healthcare. However, when providing services online, it is necessary to verify user credentials online.
[0003] In the financial sector, particularly in insurance, banking, and securities, the forgery of documents by criminals not only threatens the security of financial institutions and impacts the stability of the entire financial system, but also poses a serious threat to customer rights. For example, forging ID cards, passports, and driver's licenses for account opening, loan applications, or transactions; forging business licenses, official seals, and financial statements for corporate account opening, financing, or credit applications; forging income certificates, property certificates, and academic certificates to defraud loans or insurance claims; and potentially directly defrauding customers of funds or financial institutions' credit lines, causing substantial economic losses.
[0004] In the medical field, during hospital visits, some criminals forge ID cards and medical insurance cards to defraud medical insurance funds or obtain others' medical information. Others, unlicensed individuals forge professional qualifications to engage in medical activities, posing a significant threat to public safety. Still others, unlicensed medical institutions practice medicine illegally, causing harm to society.
[0005] In existing technologies, the review of online identification documents typically relies on staff manually verifying the authenticity of the information based on their experience, resulting in low review efficiency. Alternatively, OCR (Optical Character Recognition) can be used to automatically identify the authenticity of the document by verifying the validity of the document number. However, this method has low accuracy in identifying counterfeit documents with forged or altered codes. Summary of the Invention
[0006] In view of the shortcomings of the prior art, the present invention provides an artificial intelligence-based document recognition method, device, equipment, and medium, aiming to solve the problem that in the existing technology, when reviewing online document materials, staff members generally manually verify the authenticity of document information based on their work experience, resulting in low review efficiency. Alternatively, OCR automatic recognition is used to determine the authenticity of a document by automatically identifying whether the document number is a genuine code. However, for counterfeit documents with forged or altered codes, the recognition accuracy is low.
[0007] The technical solution of the present invention is as follows:
[0008] The first embodiment of the present invention provides an artificial intelligence-based document recognition method, the method comprising:
[0009] Obtain the ID image uploaded by the user, and extract several ID features from the ID image based on the feature extraction model;
[0010] Calculate the similarity between each document feature and the forged document vector in the forged document vector database;
[0011] Obtain the maximum similarity among several document features, and determine whether the maximum similarity is greater than a preset similarity threshold;
[0012] If the maximum similarity is greater than the preset similarity threshold, the current document is identified as a counterfeit document, and the counterfeit document information is obtained and added to the counterfeit document vector library.
[0013] If the maximum similarity is less than or equal to the preset similarity threshold, the current document will be identified as a normal document.
[0014] Another embodiment of the present invention provides an artificial intelligence-based document recognition device, the device comprising:
[0015] The feature extraction module is used to acquire the ID card image uploaded by the user and extract several ID card features from the ID card image based on the feature extraction model;
[0016] The similarity calculation module is used to calculate the similarity between each document feature and the forged document vector in the forged document vector library;
[0017] The judgment module is used to obtain the maximum similarity among several document features and to determine whether the maximum similarity is greater than a preset similarity threshold.
[0018] The identification and addition module is used to identify the current document as a counterfeit document if the maximum similarity is greater than a preset similarity threshold, obtain the counterfeit document information, and add the counterfeit document information to the counterfeit document vector library.
[0019] The determination module is used to identify the current document as a normal document if the maximum similarity is less than or equal to a preset similarity threshold.
[0020] Another embodiment of the present invention provides a computer device, the computer device including at least one processor; and,
[0021] A memory communicatively connected to the at least one processor; wherein,
[0022] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the steps of the artificial intelligence-based document recognition method described above.
[0023] Another embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the above-described artificial intelligence-based document recognition method.
[0024] Beneficial Effects: The document recognition method, apparatus, device, and medium based on artificial intelligence according to embodiments of the present invention include: acquiring a document image uploaded by a user; extracting several document features from the document image based on a feature extraction model; calculating the similarity between each document feature and a forged document vector in a forged document vector library; obtaining the maximum similarity among the several document features; determining whether the maximum similarity is greater than a preset similarity threshold; if the maximum similarity is greater than the preset similarity threshold, then identifying the current document as a forged document, acquiring forged document information, and adding the forged document information to the forged document vector library; if the maximum similarity is less than or equal to the preset similarity threshold, then identifying the current document as a normal document. Embodiments of the present invention can quickly add newly discovered forged document information to the vector library, taking effect in real time. The operation is simple, and compared to simple model optimization methods, it eliminates various tasks such as iterative training, testing, and release, saving time and money. Attached Figure Description
[0025] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0026] Figure 1 This is a schematic diagram illustrating the application environment of an embodiment of the document recognition method based on artificial intelligence according to the present invention;
[0027] Figure 2 This is a flowchart of a preferred embodiment of an artificial intelligence-based document recognition method according to the present invention;
[0028] Figure 3 This is a schematic diagram of the functional modules of a preferred embodiment of an artificial intelligence-based document recognition device of the present invention;
[0029] Figure 4This is a schematic diagram of a preferred embodiment of a computer device according to the present invention;
[0030] Figure 5 This is another structural schematic diagram of a preferred embodiment of a computer device according to the present invention. Detailed Implementation
[0031] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention is further described in detail below. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0032] The embodiments of the present invention will be described below with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.
[0033] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Here, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.
[0034] The method provided in this application can be applied to artificial intelligence (AI) scenarios. AI is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine capable of reacting in a way similar to human intelligence. Artificial intelligence studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making functions. Research in the field of artificial intelligence includes robotics, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, and fundamental AI theories.
[0035] The document recognition method based on artificial intelligence provided in this invention can be applied to, for example... Figure 1In this application environment, the client communicates with the server via a network. The client accesses the server's network or business platform, and the server can obtain the document image uploaded by the user, extract several document features from the document image based on a feature extraction model; calculate the similarity between each document feature and the forged document vector in the forged document vector library; obtain the maximum similarity among the several document features, and determine whether the maximum similarity is greater than a preset similarity threshold; if the maximum similarity is greater than the preset similarity threshold, the current document is identified as a forged document, and the forged document information is obtained and added to the forged document vector library; if the maximum similarity is less than or equal to the preset similarity threshold, the current document is identified as a normal document. In this invention, newly discovered forged information can be quickly added to the vector library, taking effect in real time. The operation is simple, eliminating various tasks such as iterative training, testing, and release, saving time and money. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.
[0036] To address the aforementioned problems, embodiments of the present invention provide an artificial intelligence-based document recognition method. Please refer to [link to relevant documentation]. Figure 2 , Figure 2 This is a flowchart illustrating a preferred embodiment of an artificial intelligence-based document recognition method according to the present invention. Figure 2 As shown, it includes:
[0037] Step S100: Obtain the ID image uploaded by the user, and extract several ID features from the ID image based on the feature extraction model.
[0038] This invention is used to identify the authenticity of documents during online business transactions. These documents include, but are not limited to, ID cards, vehicle registration certificates, passports, business licenses, property ownership certificates, academic certificates, and medical practice certificates. On the business platform, if a user uploads an image of a document, a feature extraction model is used to extract document features from the image. The document image contains several document features.
[0039] Feature extraction models are a core component of machine learning and deep learning. Their role is to automatically extract meaningful, high-level feature representations from raw data (such as images, text, and audio), providing a foundation for subsequent tasks. Feature extraction models map raw data to low- or high-dimensional feature spaces through nonlinear transformations, reducing data redundancy and computational complexity. They extract task-sensitive features to improve model performance and decompose data into independent or low-correlation feature components. Based on whether they rely on labeled data and the learning method, feature extraction models can be categorized into four types: 1. Traditional feature models: These manually design feature extraction rules based on domain knowledge, offering high computational efficiency but poor generalization. Typical methods include: Image domain: SIFT (Scale Invariant Feature Transform), HOG (Histogram of Oriented Gradients), LBP (Local Binary Pattern). Text domain: TF-IDF, N-gram, Word2Vec (requires pre-trained word vectors). Audio domain: MFCC (Mel-frequency cepstral coefficients), Short-Time Energy. 2. Supervised learning feature extraction models: These learn feature representations from labeled data, where features are strongly correlated with the task. Typical methods: Fully connected networks: such as the hidden layer outputs of a Multilayer Perceptron (MLP) as features. Convolutional Neural Networks (CNNs): such as the intermediate layer outputs of VGG and ResNet (e.g., pool5, fc7) for image classification. Recurrent Neural Networks (RNNs): such as the hidden states of LSTMs for sequence modeling (e.g., text sentiment analysis). III. Unsupervised / Self-supervised learning feature extraction models. No labeled data is required; features are learned through reconstruction, comparison, or generation tasks. Typical methods: Autoencoders: Learn compressed representations of data through an encoder-decoder structure (e.g., denoising autoencoders). Contrastive learning: such as SimCLR and MoCo, learn features by maximizing the similarity of positive sample pairs. Generative Adversarial Networks (GANs): The intermediate layer outputs of the generator can extract data distribution features (e.g., style transfer). IV. Transfer learning feature extraction models. Utilize pre-trained models (e.g., CNNs trained on ImageNet) to extract general features, fine-tuning them to adapt to new tasks. Typical methods: Fixed feature extractor: Directly use the intermediate layer outputs of the pre-trained model (e.g., the avg_pool layer of ResNet). Fine-tuning: Unfreeze some layer parameters and continue training on new data (e.g., BERT for text classification).
[0040] Specifically, step S100, which involves obtaining the user-uploaded ID image and extracting several ID features from the ID image based on a feature extraction model, includes:
[0041] Step S10: Construct an initial feature extraction model based on a deep learning network;
[0042] Step S20: Obtain a dataset of counterfeit documents, label the counterfeit documents in the dataset, and generate a sample set of counterfeit documents;
[0043] Step S30: Train the initial feature extraction model based on the forged document sample set to generate the feature extraction model.
[0044] After constructing an initial feature extraction model based on a deep learning network, a dataset is obtained. The method for constructing the forged document dataset includes simulated tampering generation. Specifically, tampering can be simulated on real document images through operations such as erasing, adding, and replacing to generate tampered samples. For example, OpenCV and PIL libraries can be used to adjust text positions, modify key information (such as names and ID numbers), or overlay forged seals. In some other embodiments, synthetic data generation methods can also be used. Specifically, the Faker library can be used to generate structured data (such as ID numbers and addresses), which can then be combined with document templates to synthesize forged samples. Faker supports multi-language and multi-field random data generation, making it suitable for constructing large-scale forged datasets. Furthermore, deepfake technology can be used, employing StyleGAN, ProGAN, and other generative adversarial networks to synthesize highly realistic forged document images. For example, a fake ID photo can be generated by training a GAN model, and forged information can be added using text synthesis techniques.
[0045] After acquiring the dataset and labeling it according to the application scenario, a sample set of counterfeit documents is generated. The initial feature extraction model is then trained based on this sample set, resulting in the final feature extraction model.
[0046] Step S20, namely obtaining a dataset of counterfeit documents, labeling the counterfeit documents in the dataset, and generating a sample set of counterfeit documents, includes:
[0047] Step S21: Obtain the dataset of forged documents;
[0048] Step S22: Label the directions of the forged documents in the forged document dataset and generate direction label values;
[0049] Step S23: Mark the corner points of the counterfeit documents in the counterfeit document dataset and generate corner point annotation values;
[0050] Step S24: Annotate the counterfeit documents in the counterfeit document dataset based on the segmentation model to generate a segmentation annotation map;
[0051] Step S25: Based on the direction annotation values, corner annotation values, and segmentation annotation map, generate a sample set of counterfeit documents.
[0052] After obtaining the dataset of counterfeit documents, the counterfeit documents are labeled with natural and universal information such as their orientation, corner points, and segmentation maps.
[0053] In the fields of computer vision and image processing, labeling directions, corners, and segmentation maps are key steps in tasks such as image understanding, object detection, and semantic segmentation.
[0054] Orientation labels mark the orientation information (such as rotation angle and orientation) of targets in images, commonly used in tasks such as vehicle detection and pedestrian pose estimation. Example: In autonomous driving, the vehicle's heading (0°–360°) is labeled to assist in path planning.
[0055] Corner annotation is the process of marking points in an image that exhibit significant feature changes (such as edges or abrupt texture shifts). It is commonly used for feature matching and SLAM (Simultaneous Localization and Mapping). Example: In AR (Augmented Reality), corner points of a checkerboard pattern are labeled for camera calibration.
[0056] Segmentation labeling involves dividing an image into multiple semantic regions (such as people, vehicles, and background) and assigning a category label to each pixel for semantic segmentation tasks. Example: In medical imaging, segmenting tumor regions to aid in diagnosis.
[0057] Directional annotation methods include single-point directional annotation, using tools such as Label Img, CVAT, and LabelMe. The annotation process involves: drawing the target bounding box; adding directional attributes (such as a rotation angle field) or visually marking the direction using arrow tools. Directional annotation also includes polygonal directional annotation methods, suitable for non-rectangular targets (such as rotated text or tilted signs). Tools include VGG Image Annotator (VIA) and Supervisely. The process involves: drawing the polygon outline; adding directional attributes (such as a vector from the polygon center to a vertex).
[0058] Corner annotation methods include manual annotation using tools such as OpenCV (interactive corner detection) and Label Img (custom corner plugin). The process involves loading the image and clicking to mark corners with the mouse. Corner coordinates are then saved (e.g., [(x1,y1),(x2,y2),...]). Alternatively, automatic detection combined with manual correction can be used, employing algorithms such as Harris corner detection, Shi-Tomasi corner detection, and FAST corner detection. The process involves running a corner detection algorithm to generate initial candidate points, manually selecting correct corners, and correcting incorrect annotations.
[0059] Segmentation image annotation methods include manual annotation. Tools: Labelbox, Segment Anything Model (SAM) interactive tool, COCO Annotator. Process: Draw polygons or free curves to outline the target contour. Assign category labels (e.g., "People" or "Vehicles") to each region. Segmentation image annotation methods also include semi-automatic annotation. Algorithms: GrabCut, Interactive Segmentation (e.g., RITM algorithm). Process: The user provides rough annotations (e.g., rectangles or doodles). The algorithm generates fine-grained segmentation masks, and erroneous regions are manually corrected. In this embodiment of the invention, the segmentation image can be automatically annotated using a general segmentation model, and then the fully annotated images are selected for use.
[0060] Step S30, which involves training the initial feature extraction model based on the forged document sample set to generate the feature extraction model, includes:
[0061] Step S31: Input the forged document sample set into the initial feature extraction model to obtain the direction prediction value, corner prediction value and segmentation prediction map;
[0062] Step S32: Calculate the loss value based on the direction labeling value, corner labeling value, segmentation labeling map, direction prediction value, corner prediction value, and segmentation prediction map;
[0063] Step S33: Adjust the training parameters of the initial feature extraction model according to the loss value, and generate the feature extraction model when training is complete.
[0064] Annotated document data is used as input. The model structure includes an encoder, a decoder, and a feature module. The intermediate features are used to predict orientation and corner information, while the decoder predicts document segmentation images. One to three of the orientation, corner, and segmentation parameters can be selected for training, depending on the situation. The more annotation dimensions, the more detailed the feature values obtained. After training, only the encoder and feature module are retained, while the decoder and loss calculation parts are removed. This yields the feature extraction model.
[0065] Step S32, which involves calculating the loss value based on the direction annotation value, corner annotation value, segmentation annotation map, direction prediction value, corner prediction value, and segmentation prediction map, includes:
[0066] Step S321: Generate a direction loss value based on the direction label value and the direction prediction value;
[0067] Step S322: Generate corner loss values based on the corner annotation values and corner prediction values;
[0068] Step S323: Generate segmentation map loss value based on segmentation annotation map and segmentation prediction map;
[0069] Step S324: Generate a loss value based on the direction loss value, the corner loss value, the segmentation map loss value, and the corresponding weight coefficients.
[0070] The loss is calculated from the predicted orientation value and the orientation label value to obtain o_loss; the loss is calculated from the predicted corner value and the corner label value to obtain c_loss; the loss is calculated from the segmentation prediction image and the segmentation label to obtain m_loss. The final loss_total = a1*o_loss + a2*c_loss + a3*m_loss, where a1, a2, and a3 are pre-set coefficients, and a1+a2+a3=1, for example a1=0.3, a2=0.3, a4=0.4. The coefficient values are adjusted according to the training effect.
[0071] Step S200: Calculate the similarity between each document feature and the forged document vector in the forged document vector library.
[0072] The system generates a feature vector of the document to be detected and a database of vectors for counterfeit documents. It calculates cosine similarity or Euclidean distance and filters for highly similar samples. The system outputs a similarity ranking list and the threshold judgment results.
[0073] Step S200, which involves calculating the similarity between each document feature and the forged document vectors in the forged document vector database, further includes:
[0074] Step S201: Respond to the forged feature vector upload instruction and obtain the image of the forged document to be identified;
[0075] Step S202: Input the forged document image into the feature extraction model to obtain the extracted forged document features;
[0076] Step S203: Compare the extracted features of the counterfeit document with the features in the counterfeit document vector library;
[0077] Step S204: If there are no forged document features in the forged document vector library, then upload the forged document features to the forged document vector library.
[0078] If a new forged document is discovered, it will be uploaded through the management console. The system will extract features from the uploaded forged document and then check if it already exists in the vector library. If it does not exist, it will insert it into the vector.
[0079] It should be noted that not all forged documents need to be added to the management console. Forged documents with the same template, that is, forged documents with the same template but different content, only need to be uploaded once.
[0080] This management console allows administrators to upload documents themselves. Uploaded documents take effect immediately, and the system will also alert administrators if an existing forged document template is used.
[0081] Step S300: Obtain the maximum similarity among several document features, and determine whether the maximum similarity is greater than a preset similarity threshold. If the maximum similarity is greater than the preset similarity threshold, then proceed to step S400. If the maximum similarity is less than or equal to the preset similarity threshold, then proceed to step S500.
[0082] Calculate the similarity between each document feature and the counterfeit document vector library, and record the maximum similarity as the maximum similarity. Determine whether the maximum similarity is greater than a preset similarity threshold. The similarity threshold can be set as needed, for example, to 50%. Based on the relationship between the maximum similarity and the similarity threshold, execute the corresponding steps.
[0083] Step S400: Identify the current document as a counterfeit document, obtain the counterfeit document information, and add the counterfeit document information to the counterfeit document vector library.
[0084] When users upload identification documents for securities account opening, bank loans, insurance claims, and medical insurance reimbursement, the system extracts features from the uploaded documents and calculates their similarity with counterfeit document vectors in a vector library. The similarity is then ranked, and the system obtains the maximum similarity score (ts1). If ts1 > a threshold, a counterfeit document is considered matched, and the uploaded document is deemed counterfeit; otherwise, it is considered legitimate. Simultaneously, the system updates the corresponding position template in the vector library with the matched counterfeit information, including time, location, and forgery details, providing data support for RAG risk assessment. RAG (Retrieval-Augmented Generation) is a technical framework combining information retrieval and text generation, aiming to improve the accuracy, reliability, and professionalism of generative models (such as large language models) by introducing external knowledge sources. RAG achieves knowledge injection through a three-stage process: retrieval, augmentation, and generation. In the retrieval stage, user input (e.g., a question) is converted into retrieval queries (e.g., keywords, semantic vectors). Retrieve relevant text fragments from structured databases (such as knowledge graphs), unstructured documents (such as PDFs and web pages), or vector databases (such as FAISS and Chroma). Filter the top-K results based on relevance scores (such as BM25 and cosine similarity). The enhancement stage is as follows: Context integration: Concatenate the retrieved text fragments with the original query to form an expanded input incorporating external knowledge. Format optimization: Add separators (such as \n\n###Retrieved Documents:\n) or prompts (such as "Answer based on the following information") to guide the model to focus on the retrieved content. The generation stage is as follows: Model inference: Feed the enhanced input into the generative model (such as GPT and Llama) to output an answer incorporating external knowledge. Post-processing: Filter duplicate content, correct syntax errors, or adjust formatting (such as generating lists or tables).
[0085] Step S500: Recognize the current document as a normal document.
[0086] When the maximum similarity is less than the similarity threshold, it means that the current document is not counterfeit, and the current document is directly output as a normal document to the visualization interface.
[0087] Step S500, which involves identifying the current document as a counterfeit document if the maximum similarity is greater than a preset similarity threshold, obtaining counterfeit document information, and adding the counterfeit document information to the counterfeit document vector library, further includes:
[0088] Step S601: Obtain the user's forged document query instruction, extract query keywords and query forged document images;
[0089] Step S602: Based on the feature extraction model, perform feature extraction on the image of the forged document to generate forged features;
[0090] Step S603: Based on the query forgery features, obtain forgery information associated with the query forgery features in the vector library;
[0091] Step S604: Input the query keywords and the forgery information into the large language model to generate a risk assessment report for the query for forged document images.
[0092] By utilizing the analysis and prediction functions of the large language model, when managers want to query the risks associated with a certain forged document, they can input the keywords they want to query along with the forged document on the management platform. After extracting features from the uploaded forged document, the system queries the vector library for various updated forgery information. Combining the query keywords input by the managers, the large language model outputs reports on risk assessments and business optimizations related to this forged document.
[0093] Large Language Models (LLMs) are artificial intelligence models based on deep learning technology capable of processing and understanding natural language. Their core objective is to simulate human language generation and comprehension abilities through training on massive amounts of text data, enabling them to perform various tasks such as text generation, translation, and question answering. Large language models typically contain billions to hundreds of billions of parameters (e.g., GPT-3 with 175 billion parameters and PaLM2 with 540 billion parameters). The number of parameters determines the model's capacity and learning ability. More parameters allow the model to capture more complex language patterns, but also increase training and inference costs.
[0094] Large language models are based on the Transformer architecture. Relying on self-attention and parallel computing capabilities, the Transformer architecture enables models to efficiently process long text sequences and capture long-distance dependencies (such as referential relationships in sentences). The self-attention mechanism achieves the ability to "find key points" by dynamically evaluating the importance of each word in the input text, while the multi-head attention mechanism understands the text from multiple perspectives, enhancing the model's comprehensiveness.
[0095] The multi-stage training process is as follows: Pre-training: Learn general language patterns on unlabeled data through masked language modeling (e.g., BERT) or autoregressive prediction (e.g., GPT). Fine-tuning: Optimize on a small scale of labeled data for specific tasks (e.g., question answering, text classification). RLHF (Reinforcement Learning Based on Human Feedback): Adjust the model output based on human feedback to make it more in line with human values (e.g., dialogue optimization in ChatGPT).
[0096] The model captures long-distance dependencies by dynamically evaluating the importance of each word in the input text. For example, in translation tasks, the model can focus on the context most relevant to the current word, avoiding the sequential processing bottleneck of traditional RNN / LSTM. Unlike traditional RNN / LSTM, the Transformer can process all tokens simultaneously, significantly improving training efficiency. For example, pre-training GPT-3 requires thousands of GPUs for parallel computation for several weeks. The model maps text to a high-dimensional semantic space, making semantically similar texts closer in the vector space. This feature supports tasks such as text similarity calculation and semantic search.
[0097] The model can be applied to Natural Language Processing (NLP). Examples include: Text generation: automatically writing press releases, advertising copy, and creative stories; Machine translation: providing high-quality cross-language translation services; Question answering systems: building intelligent customer service and teaching assistants; Sentiment analysis: analyzing the sentiment of user reviews (e.g., product review analysis on e-commerce platforms). In multimodal fusion, it can combine image, speech, and other multimodal data to expand application boundaries. Examples include: Visual question answering: answering questions based on images (e.g., VisualBERT); Speech generation: converting text into natural speech (e.g., Tacotron2); Cross-modal retrieval: searching for images or videos through text (e.g., the CLIP model).
[0098] Compared to existing technologies, the embodiments of this invention allow for the rapid registration and immediate activation of newly discovered fraudulent templates through the management platform. The operation is simple and can be completed by the organization's personnel themselves. Compared to simple model optimization methods, it eliminates various tasks such as iterative training, testing, and release, saving time, effort, and money.
[0099] During model training, labels that are universal and naturally present on various certificates are used as annotations, saving annotation workload.
[0100] During the process of matching fraudulent templates, fraudulent information such as time, location, and request information is recorded. Based on the template characteristics and the various recorded information, and in conjunction with RAG technology, fraudulent behavior is analyzed and predicted, and risk assessment and reasonable optimization suggestions are provided for business processes.
[0101] It should be noted that there is no necessary order between the above steps. Those skilled in the art will understand from the description of the embodiments of the present invention that the above steps may have different execution orders in different embodiments, that is, they may be executed in parallel or in turn, etc.
[0102] Another embodiment of the present invention provides an artificial intelligence-based document recognition device, which corresponds one-to-one with the artificial intelligence-based document recognition method described in the above embodiments. For example... Figure 3As shown, device 1 includes:
[0103] The feature extraction module 100 is used to acquire the document image uploaded by the user and extract several document features from the document image based on the feature extraction model.
[0104] The similarity calculation module 200 is used to calculate the similarity between each document feature and the forged document vector in the forged document vector library;
[0105] The judgment module 300 is used to obtain the maximum similarity among several document features and to determine whether the maximum similarity is greater than a preset similarity threshold.
[0106] The identification and addition module 400 is used to identify the current document as a counterfeit document if the maximum similarity is greater than a preset similarity threshold, obtain the counterfeit document information, and add the counterfeit document information to the counterfeit document vector library.
[0107] The determination module 500 is used to identify the current document as a normal document if the maximum similarity is less than or equal to a preset similarity threshold.
[0108] For specific implementation details, please refer to the method embodiment; they will not be repeated here.
[0109] In one embodiment, the apparatus further includes a model training module, which is specifically used for:
[0110] Construct an initial feature extraction model based on a deep learning network;
[0111] Obtain a dataset of counterfeit documents, label the counterfeit documents in the dataset, and generate a sample set of counterfeit documents;
[0112] The initial feature extraction model is trained based on the forged document sample set to generate the feature extraction model.
[0113] For specific implementation details, please refer to the method embodiment; they will not be repeated here.
[0114] In one embodiment, the model training module is also used for:
[0115] Obtain a dataset of forged documents;
[0116] The directions of the forged documents in the forged document dataset are labeled, and direction labeling values are generated.
[0117] The corner points of the counterfeit documents in the counterfeit document dataset are labeled, and corner point label values are generated;
[0118] The forged documents in the forged document dataset are labeled based on the segmentation model to generate a segmentation and labeling map.
[0119] Based on the directional annotation values, corner annotation values, and segmentation annotation map, a sample set of counterfeit documents is generated.
[0120] For specific implementation details, please refer to the method embodiment; they will not be repeated here.
[0121] In one embodiment, the model training module is also used for:
[0122] The forged document sample set is input into the initial feature extraction model to obtain the direction prediction value, corner prediction value and segmentation prediction map;
[0123] The loss value is calculated based on the orientation label value, corner label value, segmentation label map, orientation prediction value, corner prediction value, and segmentation prediction map;
[0124] The training parameters of the initial feature extraction model are adjusted based on the loss value, and the feature extraction model is generated when training is complete.
[0125] For specific implementation details, please refer to the method embodiment; they will not be repeated here.
[0126] In one embodiment, the apparatus further includes a feature comparison module, which is specifically used for:
[0127] In response to the command to upload forged feature vectors, obtain the image of the forged document to be identified;
[0128] The forged document image is input into the feature extraction model to obtain the extracted forged document features;
[0129] The extracted features of the counterfeit document are compared with the features in the counterfeit document vector library;
[0130] If the forged document vector library does not contain forged document features, then the forged document features are uploaded to the forged document vector library.
[0131] For specific implementation details, please refer to the method embodiment; they will not be repeated here.
[0132] In one embodiment, the model training module is also used for:
[0133] A direction loss value is generated based on the aforementioned direction labeling value and direction prediction value;
[0134] A corner loss value is generated based on the corner annotation value and the corner prediction value;
[0135] Generate segmentation map loss value based on segmentation annotation map and segmentation prediction map;
[0136] A loss value is generated based on the direction loss value, the corner loss value, the segmentation map loss value, and the corresponding weight coefficients.
[0137] For specific implementation details, please refer to the method embodiment; they will not be repeated here.
[0138] In one embodiment, the apparatus further includes a query and processing module, which is specifically used for:
[0139] Obtain the user's forged document query command, extract query keywords and forged document images;
[0140] Based on the feature extraction model, feature extraction is performed on the image of the forged document to generate forged query features;
[0141] Based on the query forgery features, obtain forgery information associated with the query forgery features from the vector library;
[0142] Input the query keywords and the forged information into the large language model to generate a risk assessment report for the query for forged document images.
[0143] For specific implementation details, please refer to the method embodiment; they will not be repeated here.
[0144] This invention provides an artificial intelligence-based document recognition device. Newly discovered document forgery information can be quickly added to the vector library and take effect in real time. The operation is simple. Compared with simple model optimization methods, it eliminates various tasks such as iterative training, testing, and release, saving time and money.
[0145] Another embodiment of the present invention provides a computer device, which may be a server, and its internal structure diagram may be as follows. Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a server-side artificial intelligence-based document recognition method.
[0146] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements client-side functions or steps of an artificial intelligence-based document recognition method.
[0147] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:
[0148] Obtain the ID image uploaded by the user, and extract several ID features from the ID image based on the feature extraction model;
[0149] Calculate the similarity between each document feature and the forged document vector in the forged document vector database;
[0150] Obtain the maximum similarity among several document features, and determine whether the maximum similarity is greater than a preset similarity threshold;
[0151] If the maximum similarity is greater than the preset similarity threshold, the current document is identified as a counterfeit document, and the counterfeit document information is obtained and added to the counterfeit document vector library.
[0152] If the maximum similarity is less than or equal to the preset similarity threshold, the current document will be identified as a normal document.
[0153] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:
[0154] Obtain the ID image uploaded by the user, and extract several ID features from the ID image based on the feature extraction model;
[0155] Calculate the similarity between each document feature and the forged document vector in the forged document vector database;
[0156] Obtain the maximum similarity among several document features, and determine whether the maximum similarity is greater than a preset similarity threshold;
[0157] If the maximum similarity is greater than the preset similarity threshold, the current document is identified as a counterfeit document, and the counterfeit document information is obtained and added to the counterfeit document vector library.
[0158] If the maximum similarity is less than or equal to the preset similarity threshold, the current document will be identified as a normal document.
[0159] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.
[0160] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.
[0161] The embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0162] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general-purpose hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can exist in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.
[0163] It should be noted that if any software tools or components not belonging to our company appear in the embodiments of this application, they are merely for illustrative purposes and do not represent actual use.
[0164] Among other things, conditional language such as “can,” “may,” “may,” or “may,” unless otherwise specifically stated or otherwise understood as in the context in which they are used, is generally intended to convey that a particular implementation may include (but not others) certain features, elements, and / or operations. Therefore, such conditional language is also generally intended to imply that features, elements, and / or operations are necessary for one or more implementations in any way, or that one or more implementations must include logic for determining, with or without input or prompting, whether such features, elements, and / or operations are included or will be performed in any particular implementation.
[0165] The contents already described herein in this specification and accompanying drawings include examples of methods and apparatuses capable of providing artificial intelligence-based document recognition. It is certainly not possible to describe every conceivable combination of elements and / or methods for the purpose of describing the various features of this disclosure, but it will be appreciated that many other combinations and substitutions of the disclosed features are possible. Therefore, it will be apparent that various modifications can be made to this disclosure without departing from the scope or spirit of this disclosure. Furthermore, or in alternatives, other embodiments of this disclosure may become apparent from consideration of this specification and accompanying drawings and from practice of this disclosure as presented herein. It is intended that the examples presented in this specification and accompanying drawings be considered illustrative rather than restrictive in all respects. Although specific terminology is used herein, it is used in a general and descriptive sense and is not intended for limiting purposes.
Claims
1. A document recognition method based on artificial intelligence, characterized in that... The method includes: Obtain the ID image uploaded by the user, and extract several ID features from the ID image based on the feature extraction model; Calculate the similarity between each document feature and the forged document vector in the forged document vector database; Obtain the maximum similarity among several document features, and determine whether the maximum similarity is greater than a preset similarity threshold; If the maximum similarity is greater than the preset similarity threshold, the current document is identified as a counterfeit document, and the counterfeit document information is obtained and added to the counterfeit document vector library. If the maximum similarity is less than or equal to the preset similarity threshold, the current document will be identified as a normal document.
2. The document recognition method based on artificial intelligence according to claim 1, characterized in that, Before obtaining the user-uploaded ID image and extracting several ID features from the ID image based on a feature extraction model, the process includes: Construct an initial feature extraction model based on a deep learning network; Obtain a dataset of counterfeit documents, label the counterfeit documents in the dataset, and generate a sample set of counterfeit documents; The initial feature extraction model is trained based on the forged document sample set to generate the feature extraction model.
3. The document recognition method based on artificial intelligence according to claim 2, characterized in that, The step of obtaining the forged document dataset, labeling the forged documents in the forged document dataset, and generating a forged document sample set includes: Obtain a dataset of forged documents; The directions of the forged documents in the forged document dataset are labeled, and direction labeling values are generated. The corner points of the counterfeit documents in the counterfeit document dataset are labeled, and corner point label values are generated; The forged documents in the forged document dataset are labeled based on the segmentation model to generate a segmentation and labeling map. Based on the directional annotation values, corner annotation values, and segmentation annotation map, a sample set of counterfeit documents is generated.
4. The document recognition method based on artificial intelligence according to claim 3, characterized in that, The step of training the initial feature extraction model based on the forged document sample set to generate the feature extraction model includes: The forged document sample set is input into the initial feature extraction model to obtain the direction prediction value, corner prediction value and segmentation prediction map; The loss value is calculated based on the orientation label value, corner label value, segmentation label map, orientation prediction value, corner prediction value, and segmentation prediction map; The training parameters of the initial feature extraction model are adjusted based on the loss value, and the feature extraction model is generated when training is complete.
5. The document recognition method based on artificial intelligence according to claim 1, characterized in that, Before calculating the similarity between each document feature and the forged document vectors in the forged document vector database, the method further includes: In response to the command to upload forged feature vectors, obtain the image of the forged document to be identified; The forged document image is input into the feature extraction model to obtain the extracted forged document features; The extracted features of the counterfeit document are compared with the features in the counterfeit document vector library; If the forged document feature does not exist in the forged document vector library, then the forged document feature is uploaded to the forged document vector library.
6. The document recognition method based on artificial intelligence according to claim 4, characterized in that, The calculation of the loss value based on the direction label value, corner label value, segmentation label map, direction prediction value, corner prediction value, and segmentation prediction map includes: A direction loss value is generated based on the aforementioned direction labeling value and direction prediction value; A corner loss value is generated based on the corner annotation value and the corner prediction value; Generate segmentation map loss value based on segmentation annotation map and segmentation prediction map; A loss value is generated based on the direction loss value, the corner loss value, the segmentation map loss value, and the corresponding weight coefficients.
7. The document recognition method based on artificial intelligence according to claim 1, characterized in that, If the maximum similarity is greater than a preset similarity threshold, the current document is identified as a counterfeit document, and counterfeit document information is obtained. After adding the counterfeit document information to the counterfeit document vector library, the method further includes: Obtain the user's forged document query command, extract query keywords and forged document images; Based on the feature extraction model, feature extraction is performed on the image of the forged document to generate forged query features; Based on the query forgery features, obtain forgery information associated with the query forgery features from the vector library; Input the query keywords and the forged information into the large language model to generate a risk assessment report for the query for forged document images.
8. An artificial intelligence-based document recognition device, characterized in that, The device includes: The feature extraction module is used to acquire the ID card image uploaded by the user and extract several ID card features from the ID card image based on the feature extraction model; The similarity calculation module is used to calculate the similarity between each document feature and the forged document vector in the forged document vector library; The judgment module is used to obtain the maximum similarity among several document features and to determine whether the maximum similarity is greater than a preset similarity threshold. The identification and addition module is used to identify the current document as a counterfeit document if the maximum similarity is greater than a preset similarity threshold, obtain the counterfeit document information, and add the counterfeit document information to the counterfeit document vector library. The determination module is used to identify the current document as a normal document if the maximum similarity is less than or equal to a preset similarity threshold.
9. A computer device, characterized in that, The computer device includes at least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the steps of the artificial intelligence-based document recognition method according to any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the artificial intelligence-based document recognition method according to any one of claims 1-7.