Certificate information identification method, medium and equipment

Abstract

The invention discloses a certificate information identification method, a medium and equipment, and the method comprises the steps: obtaining a to-be-detected certificate image, and extracting image features and text features contained in the certificate image; determining a text positioning map corresponding to the certificate image based on the image features and the text features so as to determine the category of each information module in the certificate image; extracting information content based on the category of each information module, and outputting initial certificate information of the certificate image; and performing information correction on the initial certificate information to obtain standard certificate information of the certificate image. Through the method, the information in the certificate image can be accurately identified.

Description

Technical Field

[0001] The present application relates to the field of image information technology, and in particular to a certificate information identification method, medium and device. Background Art

[0002] In the current certificate management and information processing scenarios, the recognition of the content of authentication certificates mainly relies on traditional optical character recognition (OCR) technology. OCR technology converts the text in the image into an editable text format by segmenting and recognizing characters in the certificate image. However, this technology often has problems such as low recognition accuracy and poor adaptability to certificates in specific formats when faced with complex certificate formats, diverse font styles, and possible interference factors such as seals and stains. Summary of the invention

[0003] The present application mainly provides a certificate information identification method, medium and device to solve the problem of inaccurate certificate information identification.

[0004] In order to solve the above technical problems, a technical solution adopted in the present application is: to provide a certificate information recognition method based on deep learning, including: obtaining a certificate image to be detected, and extracting image features and text features contained in the certificate image; determining a text location map corresponding to the certificate image based on the image features and the text features to determine the category of each information module in the certificate image; extracting information content based on the category of each information module, and outputting the initial certificate information of the certificate image; performing information correction on the initial certificate information to obtain the standard certificate information of the certificate image.

[0005] Deep learning technology based on large models can automatically learn various character features and semantic information in certificate images, and has stronger recognition capabilities for complex fonts, handwriting, blurred characters and symbols, significantly improving the accuracy of certificate content recognition.

[0006] In some embodiments, the image features include global image features; the determining of a text location map corresponding to the certificate image based on the image features and the text features to determine the category of each information module in the certificate image includes: determining a certificate template matching the certificate image based on the global image features and the text features as the text location map; and determining the category of each information module in the certificate image based on the certificate template.

[0007] In some embodiments, the image features include local image features; the determining of a text location map corresponding to the certificate image based on the image features and the text features to determine the category of each information module in the certificate image includes: determining a certificate template that most closely matches the certificate image based on the local image features and the text features as the text location map; establishing a mapping relationship between each information module on the certificate template and each information module on the certificate image based on the local image features and the text features to determine the category of each information module in the certificate image.

[0008] The certificate information template corresponding to the certificate image is determined based on the extracted global image features and text features, which is the basis for accurately identifying the certificate content. Accurate text positioning based on the certificate information template can ensure that the subsequent extraction and identification of key information in the certificate is correct, avoiding misreading or omission of information due to inaccurate positioning.

[0009] In some embodiments, the information correction of the initial certificate information includes: removing incorrectly recognized characters in the initial certificate information; correcting the format of the initial certificate information; and verifying the initial certificate information based on semantic rules.

[0010] By correcting the identified initial certificate information, it is ensured that the extracted information complies with the specifications and logic of the certificate.

[0011] In some embodiments, before obtaining the certificate image to be detected and extracting the image features and text features contained in the certificate image, it also includes: obtaining training certificate images of different types, formats and years, and marking the real certificate information of each training certificate image to build an image database; generating initial certificate information corresponding to each training certificate image based on each training certificate image; and optimizing the method of generating the initial certificate information based on the difference between the initial certificate information and the corresponding real certificate information.

[0012] By learning from a large number of certificates in different formats, the model can adaptively identify the layout and structure of various certificates without the need for cumbersome template configuration and adjustment for specific certificate formats. The model can quickly adapt to and accurately identify new certificate formats or format changes, greatly improving the versatility and flexibility of the system and reducing the maintenance and time costs of the system. The model is trained using labeled certificate data and optimized to learn the character features and semantic information in the certificate image, as well as the rules and characteristics of different certificate formats.

[0013] In some embodiments, the method of generating the initial certificate information based on the difference between the initial certificate information and the corresponding real certificate information is optimized, including: rotating, resizing, cropping or adding noise to some training certificate images in the image database.

[0014] Diversifying training data through data augmentation is helpful to improve the performance and generalization ability of the model.

[0015] In some embodiments, the certificate information includes text information, field category, and format information.

[0016] In some embodiments, the acquiring the certificate image to be detected includes: converting the certificate to be detected into a corresponding image to obtain the certificate image to be detected; and denoising, enhancing and tilt correcting the certificate image to be detected.

[0017] The collected certificate images are subjected to denoising, enhancement, tilt correction and other operations to improve the image quality and clarity, providing a good foundation for subsequent character recognition.

[0018] In order to solve the above technical problem, another technical solution adopted by the present application is: providing a storage medium on which program data is stored, and when the program data is executed by a processor, the steps of the certificate information identification method as described above are implemented.

[0019] The beneficial effects of the storage medium refer to the beneficial effects of the above-mentioned certificate information identification method, which will not be repeated here.

[0020] The present application also provides a computer device, including a processor and a memory connected to each other, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the certificate information identification method as described above are implemented.

[0021] The beneficial effects of the computer device refer to the beneficial effects of the above-mentioned certificate information identification method, which will not be repeated here.

[0022] The beneficial effects of the present application are as follows: Different from the prior art, the present application discloses a certificate information identification method, medium and device. By acquiring the certificate image to be detected, and extracting the image features and text features contained in the certificate image; determining the text positioning map corresponding to the certificate image based on the image features and text features to determine the category of each information module in the certificate image; extracting the information content based on the category of each information module, and outputting the initial certificate information of the certificate image. This certificate information identification method can adaptively identify the layout and structure of various certificates without the need for cumbersome template configuration and adjustment for specific certificate formats. For newly emerging certificate formats or format changes, the model can quickly adapt and accurately identify, greatly improving the versatility and flexibility of the system, and reducing the maintenance cost and time cost of the system. The initial certificate information is corrected to obtain standard certificate information; by correcting the identified initial certificate information to obtain standard certificate information of the certificate image, it is ensured that the extracted information complies with the specifications and logic of the certificate. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work, among which: Figure 1 It is a flowchart of an embodiment of a certificate information identification method provided by this application; Figure 2 Yes Figure 1 A schematic flow chart of an embodiment of step 40 in the method shown; Figure 3 Yes Figure 1 A schematic diagram of a flow chart of an embodiment of step 50 in the method shown; Figure 4 Yes Figure 1 A schematic flow chart of another embodiment of step 50 in the method shown; Figure 5 Yes Figure 1 A schematic diagram of a flow chart of an embodiment of step 70 in the method shown; Figure 6 Yes Figure 1 A schematic flow chart of another embodiment of the method shown; Figure 7 It is a structural schematic diagram of an embodiment of a storage medium provided by the present application; Figure 8 It is a structural diagram of an embodiment of a computer device provided by the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] The terms "first", "second", "third" in the embodiments of the present application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first", "second", "third" can expressly or implicitly include at least one of the features. In the description of the present application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0026] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0027] See also Figure 1 , Figure 1 : is a flow chart of an embodiment of a certificate information identification method provided by the present application. The certificate information identification method based on deep learning includes: Step 40: Obtain the certificate image to be detected, and extract the image features and text features contained in the certificate image.

[0028] Text features are extracted by using methods such as recurrent neural networks (RNN) or convolutional neural networks (CNN) combined with attention mechanisms, learning the semantic information and contextual relevance of the text, and thus extracting meaningful text features.

[0029] Specifically, to extract the image features and text features contained in the certificate image, it is necessary to first perform text detection on the certificate image and locate the text area in the image. The detected text area is identified and converted into readable text information. Feature extraction is performed on the identified text information. A feature extraction method based on deep learning can be used to extract the semantic features and contextual relevance of the text.

[0030] Further, see Figure 2 , step 40 comprises: Step 41: Convert the certificate to be tested into a corresponding image to obtain the certificate image to be tested.

[0031] The paper image of the certificate to be tested is collected by a scanner, a digital camera or a mobile device, and converted into a unified format as the certificate image to be tested.

[0032] Step 42: De-noise, enhance and tilt correct the certificate image to be detected.

[0033] The certificate image to be tested is subjected to operations including but not limited to median filtering to remove noise, histogram equalization to enhance image contrast, and tilt correction based on Hough transform to improve image quality and facilitate understanding and learning.

[0034] Median filtering is a nonlinear signal processing technology based on sorting statistics theory. Its basic principle is to replace the value of a point in a digital image or digital sequence with the median of the values ​​of all points in a neighborhood of the point, so that the surrounding pixel values ​​are close to the true value, thereby eliminating isolated noise points.

[0035] Histogram equalization is a basic method of image enhancement, which aims to stretch the narrow grayscale range of an image to a larger range (within the entire grayscale range) to improve the contrast of the image.

[0036] Hough transform is a method for detecting straight lines in an image. It can detect straight lines in an image by transforming points in the image space into parameter space. Hough transform-based tilt correction uses this principle to detect tilted straight lines in an image, calculate their tilt angles, and finally perform rotation correction on the image based on the angle.

[0037] By performing denoising, enhancement, tilt correction and other operations on the collected certificate images, the quality and clarity of the images can be improved, providing a good foundation for subsequent character recognition.

[0038] Step 50: Determine a text location map corresponding to the certificate image based on the image features and the text features to determine the category of each information module in the certificate image.

[0039] The global features and text features of the certificate image are extracted using image processing algorithms, and then the global features and text features are fused to form a more comprehensive feature description. The fused features are used to detect the text area in the certificate through machine learning or deep learning algorithms. Based on the detected text area, a text positioning map is generated, which can clearly mark the text location in the certificate.

[0040] Specifically, a multimodal learning method, such as a multimodal deep learning model, is used to simultaneously process image and text features and learn the association between them. The image features and text features extracted in step 40 are matched in a preset text localization map to find a text localization map corresponding to the certificate image to be detected for information recognition.

[0041] Match the detected text area with the extracted text content to generate a text location map. Fine-tune the text location map based on the matching results to ensure accuracy. Use the categories of each information module in the text location map to classify the fused features to determine the categories of the corresponding information modules in the certificate image.

[0042] The text localization map is preset in the deep learning model and is built based on a common type of certificate template. It is used to locate each information module in the certificate image based on the surrounding information of the matched information.

[0043] Furthermore, the image features include global image features, see Figure 3 , step 50 further includes: Step 51: Determine a certificate template that matches the certificate image based on the global image features and text features to serve as a text positioning map.

[0044] After extracting the global image features and text features, feature matching is required to determine the certificate template that matches the certificate image. The extracted global image features and text features are combined into a feature vector, which contains all the key information of the certificate image.

[0045] Establish a basic certificate template library, which contains a variety of known certificate templates and their corresponding feature vectors. Use feature matching algorithm to match the feature vector of the certificate image to be identified with the feature vector in the template library. Common feature matching algorithms include Euclidean distance, Manhattan distance, cosine similarity, etc. According to the result of feature matching, select the template that is most similar to the feature vector of the certificate image to be identified as the matching result, that is, the text positioning map corresponding to the certificate image, which is used to determine the category of each information module in the certificate image.

[0046] Global image features are features that describe the entire certificate image as a whole. They can reflect key information such as the overall layout, color distribution, and texture patterns of the image. Global image features can be used to make a preliminary judgment on the overall layout of the certificate, such as distinguishing areas such as the title, body, and signature. Global features help identify specific color or texture patterns in the certificate, which are often associated with specific text or image areas.

[0047] Text features are features specifically used to describe text content. They can capture detailed information such as the shape, size, and spacing of characters in the text. Text feature extraction focuses on the text information in the certificate image, including the font, size, color, position, and content of the text. After the text information is extracted through the deep learning model, various text feature extraction algorithms can be used to extract the features of the text. These features can include the frequency of characters, the pattern of character combinations, the length of the text, etc.

[0048] The certificate template contains the location and layout of each information module, as well as the definition of each information module, such as certificate title, issuing authority, recipient information, validity period, etc. Based on the known certificate template, the matching and recognition of feature information in the certificate image can be completed quickly.

[0049] By continuously optimizing and improving the feature extraction and template matching process, the matching accuracy and efficiency can be improved, providing better support for information recognition of certificate images.

[0050] Step 52: Determine the category of each information module in the certificate image based on the certificate template.

[0051] According to the position, layout and definition of each information module in the certificate template, the category definition and content identification are performed on the corresponding information module of the matching certificate image.

[0052] In another case, the image features include local image features, see Figure 4 , step 50 further includes: Step 53: Determine the certificate template that most closely matches the certificate image based on the local image features and the text features to serve as a text positioning map.

[0053] Extract local feature points of the image using algorithms such as Scale Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF), or Rapid Rotation Invariant Features (ORB). Calculate descriptors of feature points, which contain information such as gradient and direction of local areas of the image. Extract keywords or phrases from the text as text features.

[0054] Specifically, a feature matching algorithm (such as FLANN or brute force matching) is used to match the local feature points of the certificate image with the feature points in the template library, and the number, distance and other information of the matching feature points are calculated as a measure of image similarity.

[0055] Use a text matching algorithm (such as cosine similarity, Jaccard similarity or semantic matching algorithm) to match the text features of the certificate image with the text features in the template library, and calculate the text similarity as another matching basis.

[0056] Based on the image similarity and text similarity, the matching degree between the certificate image and each template in the template library is comprehensively evaluated, and the certificate template with the matching degree that meets the preset standard (the highest matching degree) is selected as the text positioning map of the certificate image.

[0057] Step 54: A mapping relationship between each information module on the certificate template and each information module on the certificate image is established based on the local image features and text features to determine the category of each information module in the certificate image.

[0058] Based on the result of feature matching, a mapping relationship between each information module on the certificate template and each information module on the certificate image is established. Based on the mapping relationship, the category of each information module in the certificate image is determined, wherein the category identification can be performed based on the information module label on the template or the predefined category rule. Finally, the identified information module category is output as structured data.

[0059] Specifically, the mapping relationship may be represented as a mapping table or a mapping function, which is used to map the information modules in the certificate image to corresponding positions on the certificate template.

[0060] Step 60: Extract information content based on the category of each information module, and output initial certificate information of the certificate image.

[0061] Based on the language analysis capability of the deep learning model, the recognized characters or sentences are subjected to semantic and grammatical analysis so that the characters can be combined into standardized, meaningful words or phrases and output. Specifically, natural language processing technology is used to perform semantic analysis on the combined text to understand its meaning and contextual relationship. Based on the results of the semantic analysis, the recognition results are correctly classified and labeled, such as key information such as name, date, issuing authority, and certificate name.

[0062] The initial certificate information is the certificate information initially output by the deep learning model. The initial certificate information still needs to be optimized and corrected in the future to form certificate information with higher usability.

[0063] Specifically, the certificate information includes text information, field category and format information.

[0064] Through this process, preliminary recognition results can be obtained, providing strong support for subsequent information processing and use.

[0065] Step 70: Modify the initial certificate information to obtain standard certificate information of the certificate image.

[0066] After the information correction process, the recognition result will become more accurate and reliable. The corrected output result usually includes corrected and optimized certificate information, such as the correct name, date, number, etc.

[0067] Optionally, during the correction process, optimization suggestions are generated based on the historical correction information. The optimization suggestions may include improvement suggestions for specific error types, such as increasing training data to improve the model's recognition ability for specific characters.

[0068] Further, see Figure 5 , step 70 further includes: Step 71: Remove the incorrectly recognized characters in the initial certificate information.

[0069] Although the deep learning model has powerful character recognition capabilities, it may still make mistakes in some cases. Through a series of algorithms and techniques, such as character replacement based on contextual information and identification of similar characters, the recognized characters are further verified and corrected.

[0070] Removing misrecognized characters in the initial certificate information helps to remove misrecognitions caused by noise, blur or character similarity, and improves the accuracy of the recognition results.

[0071] Step 72: Correct the format of the initial certificate information.

[0072] Documents such as certificates often have a specific format and layout. The identified information is formatted and corrected according to these format requirements.

[0073] For example, check whether the format of key information such as dates, numbers, names, etc. complies with the specifications, such as whether the date is arranged in the order of "year-month-day", whether the numbers contain necessary commas or decimal points, etc.

[0074] By correcting formatting errors, you can ensure that the extracted information is more standardized and easier to understand.

[0075] Step 73: Verify the initial certificate information based on semantic rules.

[0076] In addition to format verification, the identified information is further verified according to semantic rules, which usually involves techniques such as dictionary matching and syntax analysis.

[0077] For example, by comparing with the predefined certificate vocabulary, it is possible to check whether the recognized text contains non-certificate related words; through grammatical analysis, it is possible to check whether the recognized sentences conform to grammatical rules and whether the words in the sentence are arranged in the correct order.

[0078] These validation steps help ensure that the extracted information complies with the specifications and logic of the certificate.

[0079] See also Figure 6 In the certificate information identification method based on deep learning, before step 40, it also includes: Step 10: Obtain training certificate images of different types, formats and years, and annotate the real certificate information of each training certificate image.

[0080] We select different types of certificates, such as professional qualification certificates, skill level certificates, certificates of different formats, such as horizontal certificates, vertical certificates, and certificates from different years, such as five years ago and ten years ago, from public channels as training certificate images. The real certificate information of the training certificate images includes the text information, field category, and format information in the certificate, and we build an image database for deep learning training.

[0081] Optionally, the image database is also updated based on newly collected training certificate images to generalize the applicable types of certificate recognition.

[0082] Step 20: Generate initial certificate information corresponding to each training certificate image based on each training certificate image.

[0083] Based on the content of the training certificate, define the field categories that need to be extracted, such as certificate number, issuing authority, issuance date, holder name, training content, etc. Use deep learning algorithms and computer vision technology to automatically identify and annotate key information in the image, and store the extracted field values ​​and format information in a structured manner, such as JSON, XML or database records.

[0084] Deep learning algorithms are applied to large model training architectures built with TensorFlow or PyTorch.

[0085] Step 30: Based on the difference between the initial certificate information and the corresponding real certificate information, the method of generating the initial certificate information is optimized.

[0086] Define a suitable loss function according to the task requirements. For character recognition tasks, commonly used loss functions include Cross-Entropy Loss or Connectionist Temporal Classification (CTC) loss. Choose a suitable optimizer, such as Adam, SGD, or RMSprop, to update the parameters of the model. Input the preprocessed certificate image data into the model, calculate the loss through forward propagation, and then update the parameters of the model through backpropagation. Repeat the training process until the performance of the model on the validation set reaches a stable level or no longer improves significantly.

[0087] Optionally, use strategies such as grid search, random search, or Bayesian optimization during training to tune the model's hyperparameters such as learning rate, batch size, number of network layers, etc.

[0088] Optionally, the training certificate images are divided into a training set and a validation set, wherein the training set is used to train the model's certificate information recognition capability, and the validation set is used to evaluate the model's performance to ensure that the model has good generalization capability.

[0089] Optionally, in the process of optimizing the method of generating the initial certificate information, regularization techniques are used to add additional penalty terms in the loss function to limit the complexity of the model and prevent overfitting.

[0090] Optionally, in the process of optimizing the method of generating the initial certificate information, a method such as Adam or RMSprop is used to dynamically adjust the learning rate according to the first-order moment estimation and the second-order moment estimation of the gradient.

[0091] Optionally, step 30 further includes rotating, resizing, cropping or adding noise to some of the training certificate images in the image database.

[0092] Increase the diversity of training data and the robustness of the model through data augmentation techniques such as rotation, scaling, cropping, and adding noise.

[0093] Data augmentation is to increase the diversity and richness of the data set by performing a series of transformations on the original data, thereby improving the generalization ability of the model.

[0094] The identified certificate information is stored in the database, which is convenient for users to query, count and analyze, and provides strong data support for certificate management and related business decisions.

[0095] See also Figure 7 , Figure 7 It is a structural diagram of an embodiment of the storage medium provided by the present application.

[0096] The storage medium 300 stores program data 310. When the program data 310 is executed by the processor, the following is achieved: Figure 1The steps of the described certificate information identification method.

[0097] The program data 310 is stored in a storage medium 300, and includes a number of instructions for enabling a network device (which may be a router, a personal computer, a server, or other network device) or a processor to execute all or part of the steps of the method described in each embodiment of the present application.

[0098] Optionally, the storage medium 300 may be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, or other medium that can store program data.

[0099] See also Figure 8 , Figure 8 It is a structural schematic diagram of an embodiment of a computer device provided by the present application.

[0100] The computer device 400 includes a processor 420 and a memory 410 which are connected to each other. The memory 410 stores a computer program. When the processor 420 executes the computer program, the steps of the certificate information identification method described above are implemented.

[0101] Different from the prior art, the present application provides a certificate information recognition method, medium and device. Through a deep learning-based framework, the image features and text features in the certificate image are accurately extracted, thereby avoiding the problem of difficulty in recognizing handwritten fonts and special fonts in traditional methods. At the same time, text positioning map matching is performed based on the extracted features to further determine the type and content of each information module in the certificate image, so that the certificate information recognition method provided by the present application can adapt to certificates of different formats, without the need for special template configuration and parameter adjustment for each certificate format, reducing the manual configuration workload during the training process. At the same time, the certificate information recognition method based on deep learning can perform semantic verification and judgment based on contextual relationships, avoiding erroneous extraction due to semantic misunderstandings, improving the accuracy of certificate information recognition, and making the output information more usable.

[0102] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the storage medium embodiment and the computer device embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0103] The present application can be used in many general or special computing system environments or configurations, such as personal computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, network PCs, minicomputers, and distributed computing environments including any of the above systems or devices.

[0104] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not performed.

[0105] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0106] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0107] The above descriptions are merely embodiments of the present application and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A certificate information identification method based on deep learning, characterized in that: include: Acquire a certificate image to be detected, and extract image features and text features contained in the certificate image; Determine a text location map corresponding to the certificate image based on the image features and the text features to determine the category of each information module in the certificate image; extracting information content based on the category of each of the information modules, and outputting initial certificate information of the certificate image; The initial certificate information is modified to obtain standard certificate information of the certificate image.

2. The certificate information identification method according to claim 1, characterized in that: The image features include global image features; The determining of a text location map corresponding to the certificate image based on the image feature and the text feature to determine the category of each information module in the certificate image includes: Determining a certificate template matching the certificate image based on the global image feature and the text feature to serve as the text positioning map; The category of each information module in the certificate image is determined based on the certificate template.

3. The certificate information identification method according to claim 1 or 2, characterized in that: The image features include local image features; The determining of a text location map corresponding to the certificate image based on the image feature and the text feature to determine the category of each information module in the certificate image includes: Determining a certificate template that most closely matches the certificate image based on the local image features and the text features as a text positioning map; A mapping relationship between each information module on the certificate template and each information module on the certificate image is established based on the local image feature and the text feature to determine the category of each information module in the certificate image.

4. The certificate information identification method according to claim 1, characterized in that: The modifying of the initial certificate information includes: Removing incorrectly recognized characters from the initial certificate information; Correct the format of the initial certificate information; The initial certificate information is verified based on semantic rules.

5. The certificate information identification method according to claim 1, characterized in that: Before obtaining the certificate image to be detected and extracting the image features and text features contained in the certificate image, the method further includes: Obtain training certificate images of different types, formats and years, and annotate the real certificate information of each training certificate image; generating initial certificate information corresponding to each of the training certificate images based on each of the training certificate images; Based on the difference between the initial certificate information and the corresponding real certificate information, the manner of generating the initial certificate information is optimized.

6. The certificate information identification method according to claim 5, characterized in that: The optimizing the method of generating the initial certificate information based on the difference between the initial certificate information and the corresponding real certificate information includes: Some of the training certificate images in the image database are rotated, resized, cropped, or noise is added.

7. The certificate information identification method according to claim 1, characterized in that: The certificate information includes text information, field category and format information.

8. The certificate information identification method according to claim 1, characterized in that: The step of obtaining the certificate image to be detected includes: Convert the certificate to be tested into a corresponding image to obtain the certificate image to be tested; The certificate image to be detected is subjected to denoising, enhancement and tilt correction.

9. A storage medium having program data stored thereon, characterized in that: When the program data is executed by a processor, the steps of the certificate information identification method according to any one of claims 1 to 8 are implemented.

10. A computer device, characterized in that: The method comprises a processor and a memory connected to each other, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the certificate information identification method according to any one of claims 1 to 8 are implemented.

Images