File content identification method and device, electronic equipment and storage medium

Abstract

The invention provides a file content identification method and device, electronic equipment and a storage medium, and relates to the technical field of file identification. The method comprises the following steps: acquiring a to-be-identified original certificate file; the original certificate file is an image file; performing direction classification and direction correction on the original certificate file, and generating a target certificate file after direction correction; inputting the target certificate file into a pre-trained target detection model, and identifying and extracting each preset category region image in the target certificate file; and inputting each preset category region image into a corresponding preset recognition model, and generating a file content recognition result corresponding to the original certificate file. According to the file content identification method, more efficient identification of the certificate file content is realized by combining multiple deep learning algorithm models, the identification efficiency of the certificate file content is improved, and meanwhile, the auditing efficiency of the certificate file content is also improved.

Description

Technical Field

[0001] This application belongs to the field of document recognition technology, specifically relating to a document content recognition method, apparatus, electronic device, and storage medium. Background Technology

[0003] In the telecommunications industry, service processing, as a core workflow, typically encompasses multiple stages including order placement, review, scheduling, and final acceptance. The processing speed of the document review stage directly impacts the overall efficiency of the service processing workflow. Traditional document review relies heavily on offline processing and manual operation. Manual methods for identifying and reviewing document content are inefficient and struggle to meet customers' demands for fast and convenient service when faced with a massive volume of orders.

[0004] Therefore, a new document content recognition scheme is needed in related technologies to improve the efficiency of document content recognition. Summary of the Invention

[0005] The technical problem to be solved by this application is to address the above-mentioned shortcomings of the existing technology by providing a document content recognition method, apparatus, electronic device and storage medium. Using this document content recognition method can improve the efficiency of document content recognition, thereby improving the efficiency of document content review.

[0006] In a first aspect, embodiments of this application provide a method for recognizing document content, including:

[0007] Obtain the original document file to be identified; the original document file is an image file.

[0008] The original document files are oriented and corrected to generate the oriented corrected target document files.

[0009] Input the target document file into the pre-trained target detection model to identify and extract images of each preset category region in the target document file;

[0010] Each preset category region image is input into its corresponding preset recognition model to generate a document content recognition result corresponding to the original document file.

[0011] In some embodiments of the first aspect, the original document file is oriented and oriented to generate an oriented-corrected target document file, including:

[0012] Input the original document file into the pre-trained orientation classification model to generate the corresponding orientation category;

[0013] Determine whether the direction category meets the preset direction category requirements;

[0014] If the orientation category does not meet the preset orientation category requirements, the orientation of the original document file is adjusted to generate the target document file so that the orientation of the target document file meets the preset orientation category requirements.

[0015] In some embodiments of the first aspect, the target document file is input into a pre-trained target detection model to identify and extract images of preset category regions in the target document file, including:

[0016] A target detection model is used to identify regions of target document documents, generating multiple preset category regions; the preset category regions are seal regions, text regions, or face regions.

[0017] An object detection model is used to extract the region images corresponding to each preset category region in the target document file, and generate the region images of each preset category region.

[0018] In some embodiments of the first aspect, images of each preset category region are input into corresponding preset recognition models to generate document content recognition results corresponding to the original document document, including:

[0019] If the preset category region image is a seal region image, then the seal region image is input into the pre-trained seal text detection model to locate and crop the seal text region in the seal region image.

[0020] Input the seal text area into a pre-trained regional character recognition model to generate the corresponding first character recognition result;

[0021] If the preset category region image is a text region image, then the text region image is input into the region character recognition model to generate the corresponding second character recognition result;

[0022] If the preset category region image is a face image, then the face image is input into the preset face recognition model to generate the corresponding face recognition result.

[0023] In some embodiments of the first aspect, the text region image includes a handwritten text region image;

[0024] Input the text region image into the region character recognition model to generate the corresponding second character recognition result, including:

[0025] Input the handwritten text region image into the region character recognition model to generate the confidence score of handwritten text recognition in the handwritten text region image;

[0026] If the confidence level of the handwritten text recognition is greater than the preset confidence threshold, then the neatness of the handwritten text is determined to meet the preset neatness requirements, and the corresponding second text recognition result is generated.

[0027] In some embodiments of the first aspect, if the confidence level of handwritten character recognition is less than or equal to a preset confidence threshold, the method further includes:

[0028] The neatness of the handwritten text does not meet the preset neatness requirements;

[0029] The user is prompted to rewrite the handwritten text in the original document.

[0030] In some embodiments of the first aspect, after inputting the images of each preset category region into the corresponding preset recognition model to generate a document content recognition result corresponding to the original document document, the method further includes:

[0031] Determine whether the document content recognition results meet the corresponding document review requirements;

[0032] If the document content recognition result is determined to meet the corresponding document review requirements, the user will be notified that the review has been approved.

[0033] If the document content recognition result is determined to be inconsistent with the corresponding document review requirements, the user will be notified that the review has failed.

[0034] Based on the same inventive concept, in a second aspect, embodiments of this application also provide a document content recognition device, comprising:

[0035] The acquisition module is used to acquire the original document file to be identified; the original document file is an image file.

[0036] The correction module is used to classify and correct the orientation of the original document files, and generate the target document files after orientation correction.

[0037] The extraction module is used to input the target document file into the pre-trained target detection model, identify and extract the images of each preset category region in the target document file;

[0038] The generation module is used to input the images of each preset category region into the corresponding preset recognition model to generate the document content recognition result corresponding to the original document file.

[0039] In some embodiments of the second aspect, the calibration module is specifically used for:

[0040] The original document file is input into the pre-trained orientation classification model to generate the corresponding orientation category; it is then determined whether the orientation category meets the preset orientation category requirements; if it is determined that the orientation category does not meet the preset orientation category requirements, the orientation of the original document file is adjusted to generate the target document file so that the orientation of the target document file meets the preset orientation category requirements.

[0041] In some embodiments of the second aspect, the extraction module is specifically used for:

[0042] An object detection model is used to identify regions in the target document file, generating multiple preset category regions. The preset category regions are seal regions, text regions, or face regions. The object detection model is used to extract the region images corresponding to each preset category region in the target document file, generating images of each preset category region.

[0043] In some embodiments of the second aspect, the generation module is specifically used for:

[0044] If the preset category region image is a seal region image, the seal region image is input into a pre-trained seal text detection model to locate and crop the seal text region in the seal region image; the seal text region is input into a pre-trained region character recognition model to generate the corresponding first character recognition result; if the preset category region image is a text region image, the text region image is input into a region character recognition model to generate the corresponding second character recognition result; if the preset category region image is a face image, the face image is input into a preset face recognition model to generate the corresponding face recognition result.

[0045] In some embodiments of the second aspect, the text region image includes a handwritten text region image;

[0046] The generation module, when inputting a text region image into a region character recognition model to generate the corresponding second character recognition result, is specifically used for:

[0047] The handwritten text region image is input into the region character recognition model to generate the confidence score of the handwritten text recognition in the handwritten text region image; if the confidence score of the handwritten text recognition is greater than the preset confidence threshold, it is determined that the neatness of the handwritten text meets the preset neatness requirement, and the corresponding second character recognition result is generated.

[0048] In some implementations of the second aspect, if the confidence level of handwritten character recognition is less than or equal to a preset confidence threshold, the generation module is further configured to:

[0049] If the neatness of the handwritten text does not meet the preset neatness requirements, prompt the user to rewrite the handwritten text in the original document.

[0050] In some embodiments of the second aspect, the apparatus further includes:

[0051] The review module is used to determine whether the document content recognition result meets the corresponding document review requirements. If the document content recognition result meets the corresponding document review requirements, the user will be notified that the review has been approved. If the document content recognition result does not meet the corresponding document review requirements, the user will be notified that the review has been rejected.

[0052] Based on the same inventive concept, in a third aspect, embodiments of this application also provide an electronic device, the electronic device comprising:

[0053] Memory and processor;

[0054] The memory stores instructions that the computer executes;

[0055] The processor executes computer execution instructions stored in memory to implement a file content recognition method as described in any of the first aspects.

[0056] Based on the same inventive concept, in a fourth aspect, embodiments of this application also provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the file content recognition method as described in any of the first aspects.

[0057] According to the document content recognition method, apparatus, electronic device, and storage medium provided in the embodiments of this application, the original document document to be recognized is oriented and oriented corrected to generate an oriented corrected target document document. The target document document is input into a pre-trained target detection model to identify and extract images of each preset category region in the target document document. Simultaneously, the images of each preset category region are input into their respective preset recognition models to generate document content recognition results corresponding to the original document document. Thus, by combining multiple deep learning algorithm models, more efficient recognition of document document content is achieved, improving the recognition efficiency of document document content and consequently improving the review efficiency of document document content. Attached Figure Description

[0058] Figure 1 This illustration shows a flowchart of a document content recognition method provided in an embodiment of this application;

[0059] Figure 2 This illustration shows another flowchart of the document content recognition method provided in an embodiment of this application;

[0060] Figure 3 This illustration shows a flowchart of a document content review process provided in an embodiment of this application;

[0061] Figure 4 This diagram illustrates the relationship between neatness and confidence level provided in an embodiment of this application.

[0062] Figure 5 This diagram illustrates a structural schematic of a document content recognition device provided in an embodiment of this application. Detailed Implementation

[0063] To enable those skilled in the art to better understand the technical solutions of this application, the application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0064] The features and exemplary embodiments of various aspects of this application will now be described in detail. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only configured to explain this application and are not configured to limit this application. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples of this application.

[0065] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0066] Current document content recognition technologies primarily rely on offline processing and manual operation. The efficiency of manually recognizing and verifying document content is low, and it struggles to meet customers' demands for fast and convenient service when faced with a large volume of orders. Therefore, a new document content recognition solution is needed to improve its efficiency.

[0067] Example 1

[0068] The document content recognition method provided in this application can be executed by a document content recognition device and an electronic device. The following description uses the example of the document content recognition method being executed by an electronic device.

[0069] like Figure 1 As shown, the document content recognition method provided in this application embodiment may include steps S101 to S104.

[0070] S101. Obtain the original document file to be identified. The original document file is an image file.

[0071] For example, the original document can be various documents, such as business licenses, identity certificates, approval documents, authorization letters, etc. The original document is then used for subsequent recognition in the form of image files. The file format of the image file is not limited in this embodiment, and various commonly used formats can be used.

[0072] For example, the acquisition method can be through a document scanning device, through other terminal devices, or through a server.

[0073] S102. Perform orientation classification and orientation correction on the original document files to generate the orientation-corrected target document files.

[0074] For example, orientation classification can be divided into several categories based on angles. For instance, it can be divided into four categories based on every 90°, or into 36 categories based on every 10°. The more categories there are, the more refined the orientation classification, thus determining a more accurate orientation of the original document and providing a better foundation for subsequent orientation correction.

[0075] For example, orientation correction generally involves adjusting the orientation of the original document to meet the requirements. For instance, if the subsequent target detection model requires the original document to be in a positive direction (i.e., with an angle of 0°), then the orientation of the original document can be adjusted to 0°.

[0076] Orientation correction can make the original document files more compatible with the input requirements of subsequent models, ensuring input standardization and providing a foundation for accurate detection of subsequent document content.

[0077] S103. Input the target document file into the pre-trained target detection model to identify and extract the images of each preset category region in the target document file.

[0078] For example, the target detection model can use the PP-YOLOE algorithm, which can improve the accuracy and efficiency of identifying and extracting images of each preset category region in the target document file.

[0079] For example, the preset category region images can be stamp region images, text region images, face region images, etc., and the specific division and settings can be made according to the actual application. Among them, text region images can include handwritten text region images, printed text region images, watermark text region images, etc. Printed text region images can include underlined printed text region images and other text regions (such as: un-underlined printed text), and other text regions do not need to be segmented and recognized.

[0080] S104. Input the images of each preset category area into the corresponding preset recognition model to generate the document content recognition result corresponding to the original document file.

[0081] For example, a preset recognition model corresponds to a preset category region image. For instance, a stamp region image corresponds to a stamp text detection model and a region text recognition model, a text region image corresponds to a region text recognition model, and a face region image corresponds to a face recognition model.

[0082] For example, the document content recognition result is related to the content to be recognized in the original document. For instance, if there are images of a seal area and a text area, the document content recognition result will be the seal text recognition result and the text recognition result. Based on the seal text recognition result, it can be determined whether the corresponding seal meets the review requirements; based on the text recognition result, it can also be determined whether the text meets the review requirements. If there are images of a text area and a face area, the document content recognition result will be the text recognition result and the face recognition result. Similarly, based on the text recognition result and the face recognition result, it can also be determined whether the document meets the review requirements.

[0083] According to the document content recognition method provided in this application, the original document document to be recognized is oriented and oriented, and oriented correction is performed to generate an oriented-corrected target document document. The target document document is input into a pre-trained target detection model to identify and extract images of each preset category region in the target document document. At the same time, the images of each preset category region are input into the corresponding preset recognition model to generate document content recognition results corresponding to the original document document. Thus, by combining multiple deep learning algorithm models, more efficient recognition of document document content is achieved, improving the recognition efficiency of document document content and, consequently, the review efficiency of document document content.

[0084] Example 2

[0085] like Figure 2 As shown, the document content recognition method provided in this application embodiment is based on the document content recognition method provided in embodiment 1 of this application, and is further described. It may include steps S201 to S205, as follows:

[0086] S201. Obtain the original document file to be identified.

[0087] The implementation of S201 in this embodiment is similar to that of S101 in the previous embodiment, and will not be described in detail here.

[0088] S202. Perform orientation classification and orientation correction on the original document files to generate the orientation-corrected target document files.

[0089] In some implementations, S202 may be specifically as follows:

[0090] The original document file is input into a pre-trained orientation classification model to generate the corresponding orientation category. The orientation classification model can be a MobileNetV1 model.

[0091] Determine whether the direction category meets the preset direction category requirements.

[0092] If the orientation category does not meet the preset orientation category requirements, the orientation of the original document file is adjusted to generate the target document file so that the orientation of the target document file meets the preset orientation category requirements.

[0093] For example, the MobileNetV1 model adopts a lightweight design, which has high efficiency and low computational cost, making it suitable for fast document orientation determination, thereby improving the efficiency of orientation classification.

[0094] For example, the preset orientation category requirement can be a positive orientation. The orientation of the original document file can be adjusted using an image orientation adjustment algorithm to generate a target document file with the orientation adjusted, wherein the orientation of the target document file conforms to the preset orientation category requirement.

[0095] S203. Use an object detection model to identify regions of the target document document and generate multiple preset category regions. The preset category regions are seal regions, text regions, or face regions.

[0096] S204. Use an object detection model to extract the region images corresponding to each preset category region in the target document file, and generate region images for each preset category region.

[0097] For example, preset category regions correspond to preset category region images; for instance, stamp regions correspond to stamp region images, text regions correspond to text region images, and face regions correspond to face region images. By employing an object detection model, the region images corresponding to each preset category region in the target document file can be identified and extracted, providing a foundation for the targeted recognition of each preset category region image in the subsequent process.

[0098] S205. Input the images of each preset category area into the corresponding preset recognition model to generate the document content recognition result corresponding to the original document file.

[0099] In some implementations, S205 may be specifically as follows:

[0100] If the preset category region image is a seal region image, then the seal region image is input into the pre-trained seal text detection model to locate and crop the seal text region in the seal region image.

[0101] Input the seal text area into a pre-trained regional character recognition model to generate the corresponding first character recognition result.

[0102] If the preset category region image is a text region image, then the text region image is input into the region character recognition model to generate the corresponding second character recognition result.

[0103] If the preset category area image is a face image, input the face image into the preset face recognition model to generate the corresponding face recognition result.

[0104] Exemplarily, the seal text detection model can adopt the DB++ (Real-Time Scene Text Detection with Differentiable Binarization++) algorithm model. The DB++ algorithm model is a segmentation-based text detection model, which can be used to locate and crop the seal text area in the seal area image, and has a high segmentation accuracy. The regional text recognition model can adopt the SVTR_HGNet model. The SVTR_HGNet model is based on the global feature extraction of Transformer and the Hierarchical Graph Network (HGNet), which can improve the accuracy and robustness of text recognition.

[0105] Exemplarily, the first text recognition result represents the recognition result of the seal text. For example, if there is the seal text of a certain enterprise in the seal, the recognition result is the seal text of that enterprise.

[0106] Exemplarily, the preset face recognition model can adopt the YOLOv7 algorithm to improve the accuracy and speed of face recognition. The face recognition result represents the result of face recognition, such as whether the face is complete, whether the face meets the requirements (clarity, business requirements, etc.). When detecting the face, the face database can also be combined to determine whether the same face can be detected, that is, the face meets the review requirements (such as identity detection).

[0107] The second text recognition result represents the recognition result of the text in the text. For example, if the text is "Party B's signature", the recognition result is the text recognition result of "Party B's signature".

[0108] In some embodiments, the text area image includes a handwritten text area image.

[0109] The process of inputting the text area image into the regional text recognition model to generate the corresponding second text recognition result can be specifically as follows:

[0110] Input the handwritten text area image into the regional text recognition model to generate the confidence of the handwritten text recognition in the handwritten text area image.

[0111] If the confidence of the handwritten text recognition is greater than the preset confidence threshold, determine that the neatness of the handwritten text meets the preset neatness requirements, and generate the corresponding second text recognition result.

[0112] In this embodiment, testing and research show that there is a correlation between the confidence level of handwritten text recognition and the neatness of handwritten text. When the confidence level is greater than the preset confidence threshold, the neatness of handwritten text is relatively high, which meets the preset neatness requirements, thereby making the recognition accuracy of the regional text recognition model higher.

[0113] The preset reliability threshold can be set to 0.8 based on the actual research results.

[0114] For example, if the confidence level of handwritten text recognition is less than or equal to a preset confidence threshold, various processes can be executed, such as re-detection, prompting the user to rewrite the handwritten text, outputting the text recognition error result, and prompting staff to manually review it.

[0115] In some implementations, if the confidence level of handwritten character recognition is less than or equal to a preset confidence threshold, the above process further includes:

[0116] The neatness of the handwritten text does not meet the preset neatness requirements.

[0117] The user is prompted to rewrite the handwritten text in the original document.

[0118] In this embodiment, by prompting the user to rewrite the handwritten text in the original document, the user can be promptly informed of any issues with the handwritten text and any errors in the handwritten text recognition can be addressed in a timely manner, thereby improving the overall recognition efficiency of the document content.

[0119] The prompting method can be voice prompts, information prompts, etc., and this embodiment does not limit it.

[0120] In some implementations, a document review process is included after S205, as detailed below:

[0121] Determine whether the document content recognition results meet the corresponding document review requirements.

[0122] If the document content recognition result is determined to meet the corresponding document review requirements, the user will be notified that the review has been approved.

[0123] If the document content recognition result is determined to be inconsistent with the corresponding document review requirements, the user will be notified that the review has failed.

[0124] For example, document review requirements can be set according to actual needs. For instance, the document review requirements can be set to meet the following requirements: the seal meets business requirements, the text content meets business requirements, and the identity verification is passed. This embodiment does not limit this.

[0125] The document content recognition method provided in this embodiment is based on multi-model collaborative recognition, which breaks down the document review task into document orientation recognition, seal recognition, text recognition and handwriting recognition: MobileNetV1 is used for orientation classification, PP-YOLOE is used for key area detection, DB++ and SVTR_HGNet are used for seal text area detection and area text recognition, respectively, and YOLOv7 is introduced for face detection to ensure that each key area can be accurately located and recognized.

[0126] This embodiment also introduces a data-driven intelligent parameter tuning strategy. Taking handwritten character recognition as an example, by performing correlation analysis between the recognition results and the algorithm confidence level, a suitable confidence level threshold is selected as a pre-judgment standard to achieve intelligent judgment and adaptive tuning of the neatness of handwritten characters, thereby significantly improving the overall accuracy and robustness of the system.

[0127] To better understand the document content recognition method provided in this application embodiment, a specific application implementation method will be described below.

[0128] like Figure 3 As shown in the image, the business documents are certificates and certificates, including business licenses, operator identification documents, agreements, letters of introduction / authorization, network access notices, and approval forms for special offers / buildings / single locations, etc. Watermark recognition, certificate content recognition, document text content recognition, and handwritten character recognition can all be categorized under text recognition. Certificate content recognition generally refers to the recognition of text within certificates.

[0129] The document content recognition method and related business review details in this embodiment are as follows:

[0130] The document content recognition method in this embodiment aims to achieve efficient recognition and intelligent verification of various major documents through the collaborative work of multiple models. This method consists of multiple subsystems, mainly including a document orientation recognition subsystem, a target detection subsystem, a seal text detection subsystem, a region text recognition subsystem, a face detection subsystem, and a handwriting recognition and neatness judgment subsystem. Each subsystem utilizes a different deep learning model for training and optimization, and is responsible for extracting different types of information. The implementation process of this embodiment is divided into two stages:

[0131] 1. Training phase: This phase mainly includes training each model and data preprocessing to ensure that each model has efficient and accurate recognition capabilities.

[0132] 2. Application Phase: In practical applications, the system automatically reviews the documents submitted by users and combines the recognition results of various models to conduct the final intelligent review and judgment.

[0133] The solution in this embodiment aims to improve the accuracy and efficiency of document verification while reducing labor costs, thereby meeting the needs of large-scale data processing and business verification.

[0134] This embodiment uses multiple deep learning algorithm models, trained and optimized for different tasks, to achieve efficient and accurate file segmentation and text extraction quality assessment. The specific models used in each subsystem and their functions are as follows:

[0135] 1. Orientation Classification Model: The MobileNetV1 algorithm is used to classify the orientation of the input image file, detecting and determining the correct orientation for subsequent processing. This model features a lightweight design, high efficiency, and low computational cost, making it suitable for fast document orientation determination.

[0136] 2. Object Detection Model: The PP-YOLOE algorithm is used for object detection training to achieve automatic recognition of key target regions. This model can effectively identify target regions in images, such as ID numbers, photo (face) regions, seal regions, watermark regions, handwritten text regions, and printed text regions, providing accurate target localization for subsequent text extraction and quality assessment.

[0137] 3. Seal Text Detection Model: The seal text detection model is trained using the DB++ algorithm, which can accurately identify the text content in the seal area. The DB++ algorithm extracts the text from the seal by analyzing the seal image in the document, providing key text information for document verification.

[0138] 4. Region Text Recognition Model (i.e., the abbreviation text recognition model in the image): The SVTR_HGNet algorithm is used to train the region text recognition model, focusing on extracting text information from regions within a document. This optimized model can handle documents with different fonts and layouts, ensuring efficient and accurate text recognition.

[0139] 5. Face Detection Model: The YOLOv7 algorithm is used to detect faces in documents. Training with a deep convolutional neural network ensures high-precision face localization. This model can detect whether a document contains valid facial information, thus enabling identity verification during document review.

[0140] 6. Handwriting Neatness Assessment: The SVTR_HGNet algorithm is used to recognize handwritten text, and the neatness is assessed based on the confidence score. This model, through deep learning of handwriting features, can evaluate the standardization of handwritten text, further improving the quality of text extraction.

[0141] The document content recognition method in this embodiment mainly involves four recognition capabilities in practical business applications: document orientation recognition, seal recognition, text recognition, and handwriting recognition. In some scenarios, it can also be used for facial recognition. The specific solution is as follows:

[0142] 1. Seal recognition solution, including the following steps:

[0143] ① Object detection: The object detection algorithm (PP-YOLOE algorithm) is used to detect the location of the seal in the image and determine the region where the seal is located. For example... Figure 3 As shown, during training, the seal color and seal content are added, so that after training, various seals (color & content) can be identified based on the object detection algorithm.

[0144] ②Region Extraction: The identified stamp area is extracted from the original image for subsequent processing.

[0145] ③ Seal text detection: Use text detection algorithms to detect the text content in the seal area and extract the text within the area.

[0146] ④ Character recognition: The extracted text region is identified using a regional character recognition model, and the text information in the seal is returned.

[0147] 2. The handwriting recognition solution includes the following steps:

[0148] ① Neatness assessment: Based on the recognition results and confidence level of the regional character recognition model, a preliminary assessment of the neatness of the handwriting is made. If the recognition confidence level is low, the user is prompted to rewrite to ensure the accuracy of subsequent recognition.

[0149] ② Text recognition: For handwritten text that has passed the neatness judgment, the text content is directly extracted using the region character recognition model.

[0150] ③ Parameter Optimization: To further improve the accuracy of neatness judgment, an experiment was first conducted to analyze the correlation between handwritten neatness and the model's output confidence level. The experiment selected 59 handwritten images, obtained their confidence levels through a region-based character recognition model, and performed a correlation analysis with neatness. For example... Figure 4 As shown in the figure, the vertical axis represents neatness, and the horizontal axis represents confidence level. The neatness ranges from 0 to 5, and the confidence level ranges from 0 to 1. The experimental results show that there is a certain positive correlation between neatness and confidence level (correlation R2 = 0.309), proving that confidence level can be used as a reference indicator for judging neatness.

[0151] To determine the optimal confidence level for pre-interception based on neatness, 10 sample data points with different neatness levels were further selected for text recognition experiments, as shown in Table 1 below (file names are for illustrative purposes only). Based on the text recognition results and the optimal value of the confidence parameter through execution analysis, a confidence threshold of 0.8 was ultimately selected as the pre-judgment standard for handwritten neatness, ensuring that only neat handwritten text is recognized.

[0152] Table 1 Sample Table of Neatness

[0153] file name Is the identification correct? Sample neatness Confidence . / txt_rec3 / dongpengchengl.jpg yes 1 0.559507549 . / txt_rec3 / dongpengcheng2.jpg yes 2 0.917158067 . / txt_rec3 / dongpengcheng3.jpg yes 3 0.92847085 . / txt_rec3 / dongpengcheng4.jpg yes 4 0.99760133 . / txt_rec3 / dongpengcheng5.jpg yes 5 0.990303993 . / txt_rec3 / lixin1.jpg no 1 0.698264837 . / txt_rec3 / lixin2.jpg yes 2 0.999320924 . / txt_rec3 / lixin3.jpg yes 3 0.999822378 . / txt_rec3 / lixin4.jpg yes 4 0.999977469 . / txt_rec3 / lixin5.jpg yes 5 0.99998486

[0154] Meanwhile, in this embodiment, after content recognition is completed, such as Figure 3 As shown, the system enters the business review process. This embodiment employs intelligent system review, and the specific process includes: identification of key document elements (e.g., seal, identity) — business rule judgment (whether it complies with business rules) — document compliance judgment (whether the document content is compliant) — intelligent output of review results.

[0155] This embodiment uses the MobileNetV1 algorithm to classify the orientation of the document image, combined with image preprocessing techniques to achieve automatic correction, ensuring standardized input and providing an accurate foundation for subsequent text detection. It uses PP-YOLOE and DB++ algorithms to detect and segment key regions (stamps, text regions) of the document, employing offline training and online matching to build an efficient region detection model, thereby improving the stability and accuracy of text extraction.

[0156] By using the SVTR_HGNet algorithm combined with confidence parameter tuning technology, neatness judgment and character recognition of handwriting were performed. The optimal recognition threshold was determined through experimental data, forming a complete document character recognition and quality assessment system.

[0157] Meanwhile, the solution in this embodiment has proven to have the characteristics of high precision, low latency and high stability in practical applications, and can meet the needs of large-scale data processing and full-business automated review.

[0158] Example 3

[0159] like Figure 5 As shown, the document content recognition device 400 provided in this application embodiment may include:

[0160] The acquisition module 401 is used to acquire the original document file to be identified. The original document file is an image file.

[0161] The correction module 402 is used to classify and correct the orientation of the original document file, and generate the target document file after orientation correction.

[0162] The extraction module 403 is used to input the target document file into the pre-trained target detection model to identify and extract images of each preset category region in the target document file.

[0163] The generation module 404 is used to input the images of each preset category region into the corresponding preset recognition model to generate the document content recognition result corresponding to the original document file.

[0164] In some implementations, the calibration module 402 is specifically used for:

[0165] The original document file is input into a pre-trained orientation classification model to generate the corresponding orientation category. It is then determined whether the orientation category meets the preset orientation category requirements. If the orientation category does not meet the preset requirements, the orientation of the original document file is adjusted to generate the target document file, ensuring that the orientation of the target document file meets the preset orientation category requirements.

[0166] In some implementations, the extraction module 403 is specifically used for:

[0167] An object detection model is used to identify regions in the target document file, generating multiple preset category regions. These preset category regions are either seal regions, text regions, or face regions. The object detection model then extracts the region images corresponding to each preset category region from the target document file, generating images for each preset category region.

[0168] In some implementations, the generation module 404 is specifically used for:

[0169] If the preset category region image is a seal region image, the seal region image is input into a pre-trained seal text detection model to locate and crop the seal text region in the seal region image. The seal text region is then input into a pre-trained region character recognition model to generate the corresponding first character recognition result. If the preset category region image is a text region image, the text region image is input into a region character recognition model to generate the corresponding second character recognition result. If the preset category region image is a face image, the face image is input into a preset face recognition model to generate the corresponding face recognition result.

[0170] In some implementations, the text region image includes a handwritten text region image.

[0171] When the generation module 404 inputs the text region image into the region character recognition model to generate the corresponding second character recognition result, it is specifically used for:

[0172] The handwritten text region image is input into the region character recognition model to generate the confidence score for handwritten text recognition in the handwritten text region image. If the confidence score for handwritten text recognition is greater than a preset confidence threshold, the neatness of the handwritten text is determined to meet the preset neatness requirement, and the corresponding second character recognition result is generated.

[0173] In some implementations, if the confidence level of handwritten character recognition is less than or equal to a preset confidence threshold, the generation module 404 is further configured to:

[0174] The system detects that the neatness of the handwritten text does not meet the preset neatness requirements. The user is prompted to rewrite the handwritten text from the original document.

[0175] In some embodiments, the document content recognition device 400 further includes:

[0176] The review module determines whether the document content recognition results meet the corresponding document review requirements. If the document content recognition results meet the requirements, the user is notified that the review has been approved. If the document content recognition results do not meet the requirements, the user is notified that the review has been rejected.

[0177] The document content recognition device provided in this application has the beneficial effects and implementation methods of the document content recognition methods provided in Embodiments 1 and 2 of this application. For details, please refer to the specific descriptions of the document content recognition methods in Embodiments 1 and 2 above. This embodiment will not repeat them here.

[0178] Example 4

[0179] This application also provides an electronic device, which is intended to be various forms of devices with data processing capabilities, such as workbenches, servers, business devices, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0180] This electronic device includes a processor and memory. The various components are interconnected via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processor processes instructions that execute within the electronic device.

[0181] The memory is the non-transitory computer-readable storage medium provided in this application. The memory stores instructions executable by at least one processor to cause at least one processor to perform the file content recognition method provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions for causing a computer to perform the file content recognition method provided in this application.

[0182] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the file content recognition method in the embodiments of this application (e.g., appendix). Figure 5 The acquisition module 401, correction module 402, extraction module 403, and generation module 404 are shown. The processor executes various functional applications and data processing of the electronic device by running non-transient software programs, instructions, and modules stored in the memory, thereby realizing the file content recognition method in the above method embodiments.

[0183] The electronic device provided in this application has the beneficial effects and implementation methods of the file content recognition methods in Embodiments 1 and 2 of this application. For details, please refer to the specific descriptions of the file content recognition methods in Embodiments 1 and 2 above. This embodiment will not repeat them here.

[0184] Example 5:

[0185] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the file content recognition method in Embodiment 1 or Embodiment 2 above.

[0186] The computer-readable storage medium provided in this application has the beneficial effects and implementation methods of the file content recognition methods in Embodiments 1 and 2 of this application. For details, please refer to the specific descriptions of the file content recognition methods in Embodiments 1 and 2 above. This embodiment will not repeat them here.

[0187] Example 6:

[0188] This embodiment also provides a computer product that, when the instructions in the computer product are executed by the processor of an electronic device, enables the electronic device to perform the signal synchronization method of the above embodiment.

[0189] The computer product provided in this application has the beneficial effects and implementation methods of the file content recognition methods in Embodiments 1 and 2 of this application. For details, please refer to the specific descriptions of the file content recognition methods in Embodiments 1 and 2 above. This embodiment will not repeat them here.

[0190] Other embodiments of the present application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the embodiments of this application that follow the general principles of the embodiments of this application and include common knowledge or customary techniques in the art not disclosed in the embodiments of this application.

[0191] It is understood that the above embodiments are merely exemplary implementations used to illustrate the principles of this application, and this application is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of this application, and these modifications and improvements are also considered to be within the scope of protection of this application.

Claims

1. A method for recognizing document content, characterized in that, include: Obtain the original document file to be identified; The original document file is an image file; The original document file is oriented and oriented, and the oriented corrected target document file is generated. The target document file is input into a pre-trained target detection model to identify and extract images of each preset category region in the target document file; Each of the preset category region images is input into the corresponding preset recognition model to generate a document content recognition result corresponding to the original document file.

2. The method according to claim 1, characterized in that, The step of performing orientation classification and orientation correction on the original document file to generate an orientation-corrected target document file includes: The original document file is input into a pre-trained orientation classification model to generate the corresponding orientation category; Determine whether the direction category meets the preset direction category requirements; If it is determined that the direction category does not meet the preset direction category requirements, the direction of the original document file is adjusted to generate the target document file so that the direction of the target document file meets the preset direction category requirements.

3. The method according to claim 1, characterized in that, The step of inputting the target document file into a pre-trained target detection model to identify and extract images of each preset category region in the target document file includes: The target detection model is used to perform region recognition on the target document document, generating multiple preset category regions; the preset category regions are seal regions, text regions, or face regions. The target detection model is used to extract the region images corresponding to each preset category region in the target document file, and generate each preset category region image.

4. The method according to claim 3, characterized in that, The step of inputting the images of each of the preset category regions into the corresponding preset recognition models to generate a document content recognition result corresponding to the original document document includes: If the preset category region image is a seal region image, then the seal region image is input into the pre-trained seal text detection model to locate and crop the seal text region in the seal region image. Input the seal text area into a pre-trained regional character recognition model to generate the corresponding first character recognition result; If the preset category region image is a text region image, then the text region image is input into the region character recognition model to generate the corresponding second character recognition result; If the preset category region image is a face image, then the face image is input into the preset face recognition model to generate the corresponding face recognition result.

5. The method according to claim 4, characterized in that, The text region image includes a handwritten text region image; The step of inputting the text region image into the region character recognition model to generate the corresponding second character recognition result includes: The handwritten text region image is input into the region text recognition model to generate the confidence score of handwritten text recognition in the handwritten text region image; If the confidence level of the handwritten text recognition is greater than the preset confidence threshold, then the neatness of the handwritten text is determined to meet the preset neatness requirement, and the corresponding second text recognition result is generated.

6. The method according to claim 5, characterized in that, If the confidence level of handwritten character recognition is less than or equal to a preset confidence threshold, the method further includes: It was determined that the neatness of the handwritten text did not meet the preset neatness requirements; The user is prompted to rewrite the handwritten text in the original document file.

7. The method according to any one of claims 1 to 6, characterized in that, After inputting the images of each of the preset category regions into the corresponding preset recognition models to generate document content recognition results corresponding to the original document file, the method further includes: Determine whether the document content recognition result meets the corresponding document review requirements; If the document content recognition result is determined to meet the corresponding document review requirements, the user will be prompted that the review has been approved. If the document content recognition result is determined to be inconsistent with the corresponding document review requirements, the user will be prompted that the review has failed.

8. A document content recognition device, characterized in that, include: The acquisition module is used to acquire the original document file to be identified; The original document file is an image file; The correction module is used to perform orientation classification and orientation correction on the original document file, and generate the orientation-corrected target document file; The extraction module is used to input the target document file into a pre-trained target detection model to identify and extract images of each preset category region in the target document file; The generation module is used to input the images of each preset category region into the corresponding preset recognition model to generate a document content recognition result corresponding to the original document file.

9. An electronic device, characterized in that, include: Memory and processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the file content recognition method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the file content recognition method as described in any one of claims 1 to 7.

Images