A deep learning-based certificate image information intelligent recognition method and system

Abstract

The application discloses a kind of based on deep learning's certificate image information intelligent identification method and system, it is related to certificate image information identification technical field, the present application includes S1, certificate image acquisition and pre-processing, S2, certificate format classification and partition scheduling, S3, partition information identification and S4, cross-regional semantic association check and iterative correction.The present application can automatically identify multiple certificate types and versions, improve compatibility and expandability;According to priority zoning detection, positioning accuracy is high, and calculation redundancy is less, improve overall identification efficiency;Robustness is strong in complex scene, improve the identification accuracy under the condition of blur, uneven illumination, slight distortion;Realize cross-field bidirectional logic check, can automatically iterative correction error, improve the reliability of identification result;Support multi-format standardization output, improve data adaptability and practicality;High degree of automation in whole process, without manual intervention, improve the identification stability and intelligent level.

Description

Technical Field

[0001] This invention relates to the field of document image information recognition technology, specifically to a method and system for intelligent recognition of document image information based on deep learning. Background Technology

[0002] With the rapid popularization of digital government services, financial account opening, and real-name verification, document image information recognition has become a necessity. Traditional document recognition methods rely heavily on fixed template matching and simple image processing, which have poor adaptability to different document formats and versions. Furthermore, they suffer from low accuracy in complex scenarios such as blurriness, uneven lighting, and slight distortion. Therefore, it is necessary to analyze a deep learning-based intelligent document image information recognition method and system.

[0003] Existing technology, such as the invention application patent with announcement number CN115713776B, discloses a general method and system for structured recognition of identity documents based on deep learning. The method includes: acquiring identity document image information and preprocessing the acquired images; inputting standardized identity document images into a text detection network to locate text instances in the image and saving the location information to a file; extracting text images from the identity document image based on the detected text instance location coordinates and inputting them into a text recognition network for recognition, and saving the recognition results after the location information; using a key text extraction network to classify the text entities in the recognition results, removing non-key types, and then saving the classification results after the recognition results; and structuring and displaying the key text extraction results.

[0004] While existing technologies can meet basic requirements for intelligent recognition of document image information based on deep learning, they also present some potential defects and challenges, specifically in the following aspects: 1. Existing technologies largely rely on fixed templates, failing to automatically distinguish between multiple types and versions of documents, exhibiting poor compatibility with new formats and batches of documents, and prone to classification errors and region positioning failures; 2. Existing technologies largely rely on fixed templates, failing to automatically distinguish between multiple types and versions of documents, exhibiting poor compatibility with new formats and batches of documents, and prone to classification errors and region positioning failures; 3. Existing technologies mostly output recognition results for a single region independently, without performing bidirectional logical verification between fields, failing to automatically detect errors such as date conflicts and information mismatches, and lacking confidence level grading and iterative correction mechanisms. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for intelligent recognition of document image information based on deep learning, which solves the problems existing in the background technology.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: The present invention provides a method for intelligent recognition of document image information based on deep learning, including S1, document image acquisition and preprocessing, S2, document format classification and partition scheduling, S3, partition information recognition and S4, cross-regional semantic association verification and iterative correction.

[0007] S1. Document Image Acquisition and Preprocessing: Acquire the original image of the document to be identified, and perform noise reduction, normalization, and tilt correction preprocessing on the original image to obtain a standardized document image.

[0008] S2. Document Format Classification and Partition Scheduling: Based on the obtained standardized document images, a general deep learning classification model is input, and the document type and corresponding version information are output. According to the document type and version information, a general deep learning detection model is scheduled to perform partition detection on the standardized document images to obtain image sub-blocks of each region.

[0009] S3. Partition Information Recognition: Based on the obtained image sub-blocks of each region, input the corresponding preset general deep learning OCR recognition model to obtain the text recognition results and corresponding confidence scores of each region.

[0010] S4. Cross-regional semantic association verification and iterative correction: Call the preset field association rule library to perform bidirectional logical interlock verification on the obtained text recognition results of each region, determine whether there is a logical conflict in the text recognition results, and divide them into three levels of high, medium and low according to the confidence level of each text recognition result, and perform result trigger judgment respectively to obtain the final recognition result.

[0011] S5. Standardized Output: Based on the final recognition result, it is mapped to the preset standardized field intermediate layer, and then the structured recognition result in the corresponding format is output according to the actual business needs to complete the intelligent recognition of document image information.

[0012] Furthermore, the specific analysis method for outputting the document type and corresponding version information is as follows: A pre-set document type and version database is established, which stores standard format features, version identifiers, and corresponding differentiation rules for various common documents. After inputting a standardized document image into a general deep learning classification model, the model extracts the global format features of the image and compares and matches them with the standard format features in the pre-set document type and version database, calculating the feature matching degree. When the feature matching degree is greater than a pre-set matching threshold, the corresponding document type is determined. Based on the version identifier contained in the image and combined with the version differentiation rules in the database, the specific version information of the document is further determined, and the document type and corresponding version information are output.

[0013] Furthermore, the general deep learning detection model is used to perform partition detection on the standardized document image to obtain image sub-blocks of each region. The specific analysis method is as follows: based on the identified document type and corresponding version information, the field partitioning topology rules matching the document are retrieved. According to the priority of the required information area - optional information area - anti-counterfeiting feature area, the coordinates of the corresponding area are input into the general deep learning detection model to locate and select the target area of ​​the standardized document image. The selected area is then cropped at the edges and normalized at the pixels to generate image sub-blocks of a single target area. Multiple areas within the same priority are detected in parallel, while those with different priorities are detected serially to obtain independent image sub-blocks of each region.

[0014] Furthermore, the specific analysis method for obtaining the text recognition results and corresponding confidence scores of each region is as follows: a general deep learning OCR recognition model corresponding to each region of various types of documents is preset. The model is pre-trained and optimized with various document samples and different scene samples. Each region's image sub-block is input into the corresponding preset OCR recognition model. The model performs text localization, character segmentation and recognition on the image sub-block, outputs the text recognition results of the corresponding region, and calculates the confidence score of the recognition results. The confidence score is comprehensively determined by character matching accuracy, text coherence and region feature fit. Finally, the text recognition results and corresponding confidence scores of each region are obtained.

[0015] Furthermore, the specific analysis method for determining whether there is a logical conflict in the text recognition results is as follows: A preset field association rule library is retrieved. The rule library stores the preset association logic, value range, and verification rules between fields in different areas of various certificates. Based on the association rules, a two-way logical interlock verification is performed on the text recognition results of each area. The values ​​of the associated fields are compared one by one to see if there is a contradiction. If there is a situation where the field value exceeds the preset range or the associated fields contradict each other, it is determined that there is a logical conflict; if all associated fields conform to the preset rules and there is no contradiction, it is determined that there is no logical conflict.

[0016] Furthermore, the text recognition results are divided into three levels—high, medium, and low—based on their confidence levels. The specific analysis method is as follows: based on different document types and business accuracy requirements, the confidence levels of the text recognition results in each region are determined one by one and matched to the corresponding level. Among them, the high confidence level corresponds to a confidence value higher than the preset high threshold. The recognition results of this level have no obvious abnormalities after preliminary verification by the field association rule library. No re-recognition operation is required, and they are directly retained as the basic data for subsequent verification and output.

[0017] Furthermore, the specific analysis method for obtaining the final recognition result by performing result-triggered judgments is as follows: high-confidence recognition results are directly retained; medium-confidence recognition results trigger local re-recognition of the corresponding image sub-blocks; and low-confidence recognition results trigger re-cropping of the corresponding region before re-recognition, thereby obtaining recognition results without logical conflicts and with confidence levels meeting preset thresholds.

[0018] Furthermore, the specific analysis method for outputting structured recognition results in the corresponding format according to actual business needs is as follows: multiple structured output formats adapted to different business scenarios are preset, each format corresponds to a preset field mapping rule, based on the recognition results mapped to the intermediate layer of standardized fields, the field mapping rule of the corresponding format is retrieved according to the actual business scenario needs, the standardized fields are converted into structured data of the corresponding format, while invalid and redundant information is removed, field identifiers are added, and structured recognition results that meet business needs are output, which are adapted to the subsequent business system docking and data call.

[0019] Furthermore, the specific analysis method for completing the intelligent recognition of document image information is as follows: after outputting a structured recognition result that meets the actual business requirements, the document image is completed from original image acquisition, preprocessing, layout classification, partition detection, information recognition, logical verification, iterative correction to standardized output. The output structured recognition result is then subjected to a final integrity verification. When there are no logical conflicts, no missing information, the confidence level meets the preset threshold, and the format adapts to the business requirements, the intelligent recognition of document image information is completed.

[0020] A second aspect of the present invention provides a system for performing the deep learning-based intelligent recognition method for document image information, characterized in that it includes: a document image acquisition and preprocessing module: acquiring the original image of the document to be recognized, and performing noise reduction, normalization, and tilt correction preprocessing on the original image to obtain a standardized document image.

[0021] Document format classification and partition scheduling module: Based on the obtained standardized document images, input a general deep learning classification model, output document type and corresponding version information, and according to the document type and version information, schedule a general deep learning detection model to perform partition detection on the standardized document images to obtain image sub-blocks of each region.

[0022] Partition Information Recognition Module: Based on the obtained image sub-blocks of each region, input the corresponding preset general deep learning OCR recognition model to obtain the text recognition results and corresponding confidence scores of each region.

[0023] Cross-regional semantic association verification and iterative correction module: calls the preset field association rule library, performs bidirectional logical interlock verification on the obtained text recognition results of each region, determines whether there is a logical conflict in the text recognition results, and divides each text recognition result into three levels: high, medium and low based on the confidence level, and performs result trigger judgment for each level to obtain the final recognition result.

[0024] Standardized output module: Based on the final recognition result, it maps to the preset standardized field intermediate layer, and then outputs the structured recognition result in the corresponding format according to the actual business needs, thus completing the intelligent recognition of document image information.

[0025] The beneficial effects of this invention are as follows: it can automatically identify multiple document types and versions, adapt to different document formats, and improve compatibility and scalability; it performs priority-based partitioned detection, resulting in high positioning accuracy and low computational redundancy, thus improving overall recognition efficiency; it exhibits strong robustness in complex scenarios, improving recognition accuracy under conditions of blurriness, uneven lighting, and slight distortion; it achieves cross-field bidirectional logical verification, automatically iteratively correcting errors and improving the reliability of recognition results; it supports standardized output in multiple formats, allowing direct integration with business systems, thus improving data adaptability and practicality; and it boasts a high degree of automation throughout the entire process, requiring no manual intervention and improving recognition stability and intelligence. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram of the implementation steps of the method of the present invention.

[0028] Figure 2 This is a schematic diagram of the system structure connection of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Reference Figure 1As shown, the present invention provides a method for intelligent recognition of document image information based on deep learning, including S1, document image acquisition and preprocessing, S2, document format classification and partition scheduling, S3, partition information recognition, and S4, cross-regional semantic association verification and iterative correction.

[0031] S1. Document Image Acquisition and Preprocessing: Acquire the original image of the document to be identified, and perform noise reduction, normalization, and tilt correction preprocessing on the original image to obtain a standardized document image.

[0032] S2. Document Format Classification and Partition Scheduling: Based on the obtained standardized document images, a general deep learning classification model is input, and the document type and corresponding version information are output. According to the document type and version information, a general deep learning detection model is scheduled to perform partition detection on the standardized document images to obtain image sub-blocks of each region.

[0033] S3. Partition Information Recognition: Based on the obtained image sub-blocks of each region, input the corresponding preset general deep learning OCR recognition model to obtain the text recognition results and corresponding confidence scores of each region.

[0034] S4. Cross-regional semantic association verification and iterative correction: Call the preset field association rule library to perform bidirectional logical interlock verification on the obtained text recognition results of each region, determine whether there is a logical conflict in the text recognition results, and divide them into three levels of high, medium and low according to the confidence level of each text recognition result, and perform result trigger judgment respectively to obtain the final recognition result.

[0035] S5. Standardized Output: Based on the final recognition result, it is mapped to the preset standardized field intermediate layer, and then the structured recognition result in the corresponding format is output according to the actual business needs to complete the intelligent recognition of document image information.

[0036] In the above embodiments, the specific analysis method for outputting the document type and corresponding version information is as follows: a pre-set document type and version database is established, which stores standard format features, version identifiers, and corresponding differentiation rules for various common documents; after inputting a standardized document image into a general deep learning classification model, the model extracts the global format features of the image and compares and matches them with the standard format features in the pre-set document type and version database to calculate the feature matching degree; when the feature matching degree is greater than a pre-set matching threshold, the corresponding document type is determined, and based on the version identifier contained in the image and combined with the version differentiation rules in the database, the specific version information of the document is further determined, and the document type and corresponding version information are output.

[0037] It should be noted that various common documents include ID cards, passports, driver's licenses, business licenses, etc.

[0038] It should be noted that version identifiers include document printing batches and format version numbers.

[0039] It should be noted that a pre-set database of document types and versions is provided. The database pre-stores the standard format features of common documents such as ID cards, passports, driver's licenses, and business licenses. These standard format features include inherent format information such as document size proportions, border styles, background textures, key field layouts, color characteristics, and anti-counterfeiting zone locations. Simultaneously, it stores the version identifiers and version differentiation rules corresponding to each document. The version identifiers include visually identifiable markings such as the format version number printed on the document, the year of issuance, the printing batch, and the issuing authority's version code. The version differentiation rules are used to uniquely determine the specific version of the document based on the position, content, and format of the version identifier.

[0040] It should be noted that the general deep learning classification model performs global feature extraction on standardized document images. The extracted features include: global texture features, edge contour features, layout structure features, key area location distribution features, and color histogram features, forming a fixed-dimensional global layout feature vector.

[0041] It should be noted that feature comparison and matching and matching degree calculation: the extracted global layout feature vector is compared with the standard layout feature vector of each document type in the database by cosine similarity calculation to obtain the feature matching degree; the feature matching degree takes a value range of 0-1, and the closer the value is to 1, the more consistent the layout features are.

[0042] It should be noted that the preset matching threshold is set by relevant staff, for example, 0.85, and can be adjusted within the range of 0.7-0.95 according to the recognition accuracy requirements; when the feature matching degree is greater than this threshold, it is determined that the match is successful and the corresponding document type is determined; otherwise, it is determined that it cannot be recognized or is not a preset document type.

[0043] It should be noted that further determination of the specific version information of the certificate is required: after determining the certificate type, the model locates and recognizes the version identifier area in the image to obtain the version identifier string; according to the preset version differentiation rules in the database, the identified version identifier is compared with the standard identifier of each version. If the match is successful, the corresponding specific version information is output, thus completing the output of the certificate type and version information.

[0044] In the above embodiments, the scheduling general deep learning detection model performs partition detection on the standardized document image to obtain image sub-blocks of each region. The specific analysis method is as follows: based on the identified document type and corresponding version information, the field partitioning topology rule matching the document is retrieved. According to the priority of the required information area - optional information area - anti-counterfeiting feature area, the coordinates of the corresponding area are input into the general deep learning detection model to locate and select the target area of ​​the standardized document image. The selected area is then cropped at the edge and normalized to generate image sub-blocks of a single target area. Multiple areas within the same priority are detected in parallel, while those with different priorities are detected serially to obtain independent image sub-blocks of each region.

[0045] It should be noted that the field partitioning topology rules are pre-configured in the system's structured configuration file, which includes: document type, version number, field area name, relative or absolute coordinates, aspect ratio, area order, and priority flag. Different versions of documents correspond to different topology rules to ensure accurate positioning for different layouts.

[0046] It should be noted that the required information area includes core identification fields such as name, number, and expiration date; the optional information area includes non-essential but identifiable fields such as address and remarks; and the anti-counterfeiting feature area includes areas used for auxiliary verification such as QR code, security line, and fluorescent area.

[0047] It should be noted that the specific analysis method for target region localization and bounding box selection of standardized document images is as follows: obtain the coordinates of the upper left corner (x1, y1) and lower right corner (x2, y2) of each region according to the field partitioning topology rules, and input them into a general deep learning detection model; the model performs target detection and bounding box regression on the standardized document images, outputs accurate bounding rectangles, and completes region localization and bounding box selection.

[0048] It should be noted that the edge cropping and pixel normalization of the selected area are performed as follows: the selected area is cropped to remove excess background and black borders; then pixel normalization is performed: the area is uniformly scaled to 64×64 or 128×128 pixels, uniformly converted to grayscale or three-channel RGB image, and the pixel values ​​are normalized to the 0-1 range to eliminate size and lighting differences.

[0049] It should be noted that multiple regions within the same priority are detected in parallel, while different priorities are detected sequentially. The specific analysis method is as follows: multiple regions within the same priority are detected in parallel using multi-threading and simultaneously fed into the model for inference; different priorities are detected sequentially, and the detection of the next priority must be started only after all the detection of the previous priority has been completed, so as to ensure that key information is processed first.

[0050] It should be noted that the final output of independent image sub-blocks are: single-field regions, interference-free, uniform in size, and standardized in format image blocks, which are dedicated to subsequent OCR recognition.

[0051] In the above embodiments, the specific analysis method for obtaining the text recognition results and corresponding confidence scores of each region is as follows: a general deep learning OCR recognition model corresponding to each region of various types of documents is preset. The model is pre-trained and optimized with various document samples and different scene samples. Each region's image sub-block is input into the corresponding preset OCR recognition model. The model performs text localization, character segmentation and recognition on the image sub-blocks, outputs the text recognition results of the corresponding regions, and calculates the confidence score of the recognition results. The confidence score is comprehensively determined by character matching accuracy, text coherence and region feature fit. Finally, the text recognition results and corresponding confidence scores of each region are obtained.

[0052] It should be noted that different scenarios include blurriness, uneven lighting, and slight distortion.

[0053] It should be noted that the general deep learning OCR recognition model adopts an end-to-end text recognition network, including: a backbone feature extraction network, such as ResNet or MobileNet, a sequence modeling module, and an output layer. The model is lightweightly designed for the characteristics of document fields to ensure both speed and accuracy.

[0054] It should be noted that the OCR model is pre-trained using a multi-type document sample set, covering: ID card, passport, driver's license, business license, etc.; and includes complex scene samples such as blurry, uneven lighting, slight distortion, tilt, and low resolution; the training adopts adaptive learning rate, early stopping strategy, and data augmentation, such as rotation, scaling, and noise superposition, to improve generalization ability.

[0055] It should be noted that text positioning means determining the position of a text line; character segmentation means splitting consecutive characters; and character recognition means outputting the recognition result character by character.

[0056] It should be noted that the confidence level described in this invention is calculated by weighting the following three parts: character matching accuracy, which is the mean of the single character recognition probability, ranging from 0 to 1; text coherence, which is the determination of whether a field is a legal field based on a dictionary / rule, such as an 18-digit ID number or a date format; and regional feature fit, which is the degree of matching between the text length, character type, and the preset rules for that region. For example: total confidence level = 0.6 × character accuracy + 0.2 × text coherence + 0.2 × regional feature fit.

[0057] It should be noted that the confidence level between 0 and 1 in the final output text recognition result of the model is used for subsequent graded verification and iterative correction.

[0058] In the above embodiments, the specific analysis method for determining whether there is a logical conflict in the text recognition results is as follows: A preset field association rule library is retrieved. The rule library stores the preset association logic, value range, and verification rules between fields in different areas of various certificates. Based on the association rules, a two-way logical interlock verification is performed on the text recognition results of each area. The values ​​of the associated fields are compared one by one to see if there is a contradiction. If there is a situation where the field value exceeds the preset range or the associated fields contradict each other, it is determined that there is a logical conflict. If all associated fields conform to the preset rules and there is no contradiction, it is determined that there is no logical conflict.

[0059] It should be noted that the values ​​of the associated fields should be compared one by one to check for contradictions, such as logical inconsistencies between the validity period and the issuance date of the certificate, or mismatches between the identity information and the certificate type.

[0060] It should be noted that the field association rule library is a preset structured rule set, which is stored according to document type. Each rule includes: document type, associated field pairs, such as issuance date-validity period, name-gender, field value range, such as length, character type, logical constraints between fields such as greater than, less than, belong to, match, conflict judgment conditions and prompt information.

[0061] It should be noted that the association logic includes, but is not limited to: date logic: expiry date > issuance date; number logic: for example, ID card number is 18 digits long and includes a check digit; matching logic: for example, the 17th digit of the ID card number matches the gender field; format logic: certificate number, date, and agency name conform to the standard format; type logic: field content matches the document type, such as passport number does not contain Chinese characters.

[0062] It should be noted that the range of values ​​includes: length range: for example, 18-digit ID number; character type: pure numbers, numbers + letters, Chinese characters + letters; format specification: the specified date format; enumeration range: gender is limited to male / female, and nationality is limited to a valid list.

[0063] It should be noted that the two-way logical interlock verification is performed in the following steps: 1. Read the recognition results of the two related fields; 2. Verify B according to rule A constraint in the rule base; 3. Verify A according to rule B; 4. If any step is not satisfied, a conflict is determined; 5. If all are satisfied, proceed to the next related pair verification.

[0064] It should be noted that a logical conflict is determined to exist if any of the following conditions are met: field length or character type does not conform to the value range, date sequence is reversed, number verification fails, related field content does not match, or content is obviously inconsistent with document type.

[0065] In the above embodiments, the text recognition results are divided into three levels: high, medium, and low, based on their confidence levels. The specific analysis method is as follows: the confidence levels of the text recognition results in each region are determined one by one according to different document types and business accuracy requirements, and the corresponding levels are matched. The high confidence level corresponds to a confidence value higher than a preset high threshold. The recognition results of this level have no obvious abnormalities after preliminary verification by the field association rule library, and no re-recognition operation is required. They are directly retained as the basic data for subsequent verification and output.

[0066] It should be noted that the confidence level of the text recognition results in each region is determined individually based on different document types and business accuracy requirements, and matched with the corresponding level. When the confidence level is greater than or equal to the maximum value of the reference confidence level safety interval, it is considered high confidence level, for example, confidence level ≥ 0.9; when the confidence level is within the reference confidence level safety interval, it is considered medium confidence level, for example, 0.7 ≤ confidence level < 0.9; when the confidence level is less than or equal to the minimum value of the reference confidence level safety interval, it is considered low confidence level, for example, < 0.7.

[0067] It should be noted that the following hierarchical steps are performed on the text recognition results of each region: 1. Obtain the confidence value of the OCR output; 2. Compare it with the high, medium and low thresholds in turn; 3. Determine the level and mark it; 4. The high confidence field enters the preliminary verification; 5. The medium and low confidence fields enter the subsequent re-recognition process.

[0068] It should be noted that the high-confidence recognition results need to be initially verified through the field association rule library. The verification content includes: the field length and character type are within the range of values; there are no obvious format errors, such as the date and number formats are valid; there are no obvious contradictions with adjacent key fields; no obvious anomalies means: valid format, no obvious errors, and no conflicts.

[0069] It should be noted that recognition results with high confidence and no abnormalities in the initial verification are directly locked and retained, and will not participate in local re-recognition or re-cropping recognition, thereby reducing redundant calculations and improving the overall recognition efficiency.

[0070] In the above embodiments, the specific analysis method for obtaining the final recognition result by performing result-triggered judgment is as follows: high-confidence recognition results are directly retained, medium-confidence recognition results trigger local re-recognition of the corresponding image sub-blocks, and low-confidence recognition results trigger re-cropping of the corresponding area before recognition, so as to obtain recognition results without logical conflicts and with confidence levels meeting the preset threshold.

[0071] It should be noted that the medium confidence recognition result triggers local re-recognition: the original region image sub-block is directly input into the corresponding OCR model again, image enhancement is enabled, including contrast enhancement and sharpening, and text localization and character recognition are performed again, so that the second result overwrites the original result, and local optimization and correction are completed.

[0072] It should be noted that low-confidence recognition results trigger re-cropping: based on the field partitioning topology rules, the region coordinates are expanded or fine-tuned, for example, by expanding outward by 5%-10% pixels, to reposition, reselect, and regenerate image sub-blocks before being sent to the OCR model for recognition, thus resolving recognition failures caused by positioning deviations.

[0073] It should be noted that the result-triggered judgment is executed according to the following process: high confidence - directly retain; medium confidence - partial re-identification - recalculate confidence; low confidence - re-prune - re-identify - recalculate confidence; all fields are re-verified for cross-regional semantic association; the process is repeated until the termination condition is met.

[0074] In the above embodiments, the specific analysis method for outputting structured recognition results in the corresponding format according to actual business needs is as follows: multiple structured output formats adapted to different business scenarios are preset, each format corresponds to a preset field mapping rule, based on the recognition results mapped to the intermediate layer of standardized fields, according to the actual business scenario needs, the field mapping rule of the corresponding format is retrieved, the standardized fields are converted into structured data of the corresponding format, and invalid and redundant information is removed, field identifiers are added, and structured recognition results that meet business needs are output, which are adapted to the subsequent business system docking and data call.

[0075] It should be noted that different business scenarios may use structured output formats, including common formats such as tables, JSON, and XML.

[0076] It should be noted that the standardized field intermediate layer has a unified field structure that does not change with the document version or output format, and includes, but is not limited to: document type, version number, name, gender, ethnicity, date of birth, address, document number, issuing authority, validity start date, validity end date, photo area, anti-counterfeiting area mark, confidence level, verification status, and recognition time.

[0077] It should be noted that the field mapping rules are preset configuration files, which achieve: a one-to-one correspondence between standardized fields and business system field names; support for custom field aliases and field order; support for required / optional field marking; and support for format conversion, such as date, encoding, and case sensitivity.

[0078] It should be noted that invalid and redundant information includes: blank fields, garbled characters, meaningless symbols, duplicate recognition content, and fields that are outside the scope of business operations. The removal methods are: automatic removal based on a preset blacklist, null value detection, and length filtering.

[0079] It should be noted that the supplementary field identifiers include: field name, field meaning, confidence score, verification status such as pass / fail, region source, number of identifications, and number of corrections.

[0080] It should be noted that the process of outputting the structured recognition results in the corresponding format is as follows: 1. Load the final recognition results into the standardized field intermediate layer; 2. Select the output format according to business requirements; 3. Call the corresponding field mapping rules; 4. Remove redundant information and supplement field identifiers; 5. Generate formatted data; 6. Output and return to the business caller.

[0081] In the above embodiments, the specific analysis method for completing the intelligent recognition of document image information is as follows: after outputting a structured recognition result that meets the actual business requirements, the document image is completed from original image acquisition, preprocessing, layout classification, partition detection, information recognition, logical verification, iterative correction to standardized output. The output structured recognition result is then subjected to a final integrity verification. When there are no logical conflicts, no missing information, the confidence level meets the preset threshold, and the format adapts to the business requirements, the intelligent recognition of document image information is completed.

[0082] It should be noted that the final integrity check is a full-field compliance check performed after the structured results are output, including: field integrity check: all required fields have been identified and are not empty; logical consistency check: all related fields are free of conflict; confidence check: all fields meet the minimum confidence requirement; format validity check: the output format is consistent with the business requirements.

[0083] It should be noted that "no missing information" means that all required key fields for the current document type contain valid and identifiable information, with no empty values, garbled characters, or unreadable fields. Required fields include: document number, name, validity period, and issuing authority, which are determined according to the document type.

[0084] It should be noted that the confidence level of all fields is greater than or equal to the lowest reference confidence level of the field; the confidence level of core required fields is greater than or equal to the highest reference confidence level of the field; and the overall average confidence level is greater than or equal to the average reference confidence level.

[0085] It should be noted that format adaptation to business requirements means that the output data format, field names, encoding methods, and field order are completely consistent with the requirements of the business system, and can be directly imported, integrated, and stored in the database without secondary processing.

[0086] It should be noted that the intelligent recognition process for document image information is considered complete only if all of the following conditions are met simultaneously: structured recognition results have been output, the final integrity check has passed, there are no logical conflicts, no missing required information, the confidence level of all fields meets the preset threshold, and the output format matches the business requirements.

[0087] Reference Figure 2As shown, the present invention provides a system for intelligent recognition of document image information based on deep learning, characterized in that it includes: a document image acquisition and preprocessing module: acquiring the original image of the document to be recognized, and performing noise reduction, normalization, and tilt correction preprocessing on the original image to obtain a standardized document image.

[0088] Document format classification and partition scheduling module: Based on the obtained standardized document images, input a general deep learning classification model, output document type and corresponding version information, and according to the document type and version information, schedule a general deep learning detection model to perform partition detection on the standardized document images to obtain image sub-blocks of each region.

[0089] Partition Information Recognition Module: Based on the obtained image sub-blocks of each region, input the corresponding preset general deep learning OCR recognition model to obtain the text recognition results and corresponding confidence scores of each region.

[0090] Cross-regional semantic association verification and iterative correction module: calls the preset field association rule library, performs bidirectional logical interlock verification on the obtained text recognition results of each region, determines whether there is a logical conflict in the text recognition results, and divides each text recognition result into three levels: high, medium and low based on the confidence level, and performs result trigger judgment for each level to obtain the final recognition result.

[0091] Standardized output module: Based on the final recognition result, it maps to the preset standardized field intermediate layer, and then outputs the structured recognition result in the corresponding format according to the actual business needs, thus completing the intelligent recognition of document image information.

[0092] The above content is merely an example and illustration of the concept of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the concept of the invention or exceed the scope defined by the present invention, and all such modifications and additions should fall within the protection scope of the present invention.

Claims

1. A method for intelligent recognition of document image information based on deep learning, characterized in that, include: S1. Document Image Acquisition and Preprocessing: Acquire the original image of the document to be identified, and perform noise reduction, normalization, and tilt correction preprocessing on the original image to obtain a standardized document image. S2, Document Format Classification and Partition Scheduling: Based on the obtained standardized document images, input a general deep learning classification model, output document type and corresponding version information, and schedule a general deep learning detection model to perform partition detection on the standardized document images according to the document type and version information to obtain image sub-blocks of each region; S3, Partition Information Recognition: Based on the obtained image sub-blocks of each region, input the corresponding preset general deep learning OCR recognition model to obtain the text recognition results and corresponding confidence scores of each region; S4. Cross-regional semantic association verification and iterative correction: Call the preset field association rule library to perform bidirectional logical interlock verification on the obtained text recognition results of each region, determine whether there is a logical conflict in the text recognition results, and divide them into three levels of high, medium and low according to the confidence level of each text recognition result, and perform result trigger judgment respectively to obtain the final recognition result. S5. Standardized Output: Based on the final recognition result, it is mapped to the preset standardized field intermediate layer, and then the structured recognition result in the corresponding format is output according to the actual business needs to complete the intelligent recognition of document image information.

2. The method for intelligent recognition of document image information based on deep learning according to claim 1, characterized in that, The specific analysis method for the output document type and corresponding version information is as follows: A pre-set database of document types and versions is provided, which stores the standard format features, version identifiers and corresponding differentiation rules of various common documents. After the standardized document image is input into a general deep learning classification model, the model extracts the global format features of the image and compares and matches them with the standard format features in the pre-set database of document types and versions to calculate the feature matching degree. When the feature matching degree is greater than the preset matching threshold, the corresponding document type is determined, and based on the version identifier contained in the image and the version differentiation rules in the database, the specific version information of the document is further determined, and the document type and corresponding version information are output.

3. The method for intelligent recognition of document image information based on deep learning according to claim 2, characterized in that, The general deep learning detection model described above performs partition detection on standardized document images to obtain image sub-blocks in each region. The specific analysis method is as follows: Based on the identified document type and corresponding version information, the field partitioning topology rules matching the document are retrieved. The coordinates of the corresponding regions are input into a general deep learning detection model according to the priority of the required information area, optional information area, and anti-counterfeiting feature area. The target region is located and selected in the standardized document image. The selected region is then cropped at the edges and normalized at the pixels to generate image sub-blocks of a single target region. Multiple regions within the same priority are detected in parallel, while those with different priorities are detected sequentially to obtain independent image sub-blocks of each region.

4. The method for intelligent recognition of document image information based on deep learning according to claim 3, characterized in that, The specific analysis method for obtaining the text recognition results and corresponding confidence scores for each region is as follows: A general deep learning OCR recognition model corresponding to each region of various types of documents is preset. The model is pre-trained and optimized with various document samples and different scene samples. Each region image sub-block is input into the corresponding preset OCR recognition model. The model performs text localization, character segmentation and recognition on the image sub-block, and outputs the text recognition result of the corresponding region. At the same time, the confidence of the recognition result is calculated. The confidence is comprehensively determined by character matching accuracy, text coherence and region feature fit. Finally, the text recognition result and corresponding confidence of each region are obtained.

5. The method for intelligent recognition of document image information based on deep learning according to claim 4, characterized in that, The specific analysis method for determining whether there is a logical conflict in the text recognition results is as follows: The system retrieves a preset field association rule library, which stores preset association logic, value ranges, and verification rules between fields in different areas of various certificates. Based on the association rules, the system performs bidirectional logical interlock verification on the text recognition results of each area, comparing the values ​​of the associated fields one by one to see if there are any contradictions. If there are cases where the field values ​​exceed the preset range or the associated fields contradict each other, it is determined that there is a logical conflict. If all associated fields conform to the preset rules and are not contradictory, then it is determined that there is no logical conflict.

6. The method for intelligent recognition of document image information based on deep learning according to claim 5, characterized in that, Simultaneously, based on the confidence level of each text recognition result, it is divided into three levels: high, medium, and low. The specific analysis method is as follows: Based on different document types and business accuracy requirements, the confidence level of the text recognition results in each region is determined one by one and matched with the corresponding level. The high confidence level corresponds to a confidence value higher than the preset high threshold. The recognition results of this level have no obvious abnormalities after preliminary verification by the field association rule library. No re-recognition operation is required, and they are directly retained as the basic data for subsequent verification and output.

7. The method for intelligent recognition of document image information based on deep learning according to claim 6, characterized in that, The specific analysis method for obtaining the final recognition result by performing result-triggered judgments is as follows: High-confidence recognition results are directly retained, medium-confidence recognition results trigger local re-recognition of corresponding image sub-blocks, and low-confidence recognition results trigger re-cropping of the corresponding region before re-recognition, resulting in recognition results without logical conflicts and with confidence levels meeting preset thresholds.

8. The intelligent recognition method for document image information based on deep learning according to claim 7, characterized in that, The specific analysis method for outputting structured recognition results in the corresponding format according to actual business needs is as follows: It has multiple preset structured output formats adapted to different business scenarios. Each format corresponds to a preset field mapping rule. Based on the recognition result mapped to the intermediate layer of standardized fields, it retrieves the field mapping rule of the corresponding format according to the actual business scenario requirements, converts the standardized fields into structured data of the corresponding format, removes invalid and redundant information, adds field identifiers, and outputs structured recognition results that meet business requirements, which can be adapted to the subsequent business system integration and data call.

9. The intelligent recognition method for document image information based on deep learning according to claim 8, characterized in that, The specific analysis method for achieving intelligent recognition of document image information is as follows: Once the output is a structured recognition result that meets the actual business requirements, the process of acquiring the original image, preprocessing, classifying the layout, detecting the partitions, recognizing the information, verifying the logic, iterating and correcting the logic, and then standardizing the output is completed. The structured recognition result is then subjected to a final integrity check. When there are no logical conflicts, no missing information, the confidence level meets the preset threshold, and the format is adapted to the business requirements, the intelligent recognition of the document image information is completed.

10. A system for performing the intelligent recognition method for document image information based on deep learning as described in any one of claims 1-9, characterized in that, include: Document image acquisition and preprocessing module: acquires the original image of the document to be identified, and performs noise reduction, normalization and tilt correction preprocessing on the original image to obtain a standardized document image; Document format classification and partition scheduling module: Based on the obtained standardized document images, input a general deep learning classification model, output document type and corresponding version information, and according to the document type and version information, schedule a general deep learning detection model to perform partition detection on the standardized document images to obtain image sub-blocks of each region; Partition information recognition module: Based on the obtained image sub-blocks of each region, input the corresponding preset general deep learning OCR recognition model to obtain the text recognition results and corresponding confidence scores of each region; Cross-regional semantic association verification and iterative correction module: calls the preset field association rule library, performs bidirectional logical interlock verification on the obtained text recognition results of each region, determines whether there is a logical conflict in the text recognition results, and divides each text recognition result into three levels: high, medium and low based on the confidence level, and performs result trigger judgment for each level to obtain the final recognition result. Standardized output module: Based on the final recognition result, it maps to the preset standardized field intermediate layer, and then outputs the structured recognition result in the corresponding format according to the actual business needs, thus completing the intelligent recognition of document image information.

Images