Complex text OCR (Optical Character Recognition) error recognition and repair method based on large language model

Abstract

The invention discloses a complex text OCR (Optical Character Recognition) error recognition and repair method based on a large language model, and relates to the technical field of text processing, the method comprises the following steps: step 1, a text preprocessing module recognizes and excludes a non-text area in a preliminary text result generated by OCR to ensure that residual content is pure text input, and a text preprocessing module is used for preprocessing the residual content; obtaining a text result of the plain text data; step 2, an error recognition module performs entity recognition on a text result of the plain text data by using a large language model, and provides revision suggestions for recognized suspected errors; 3, the database provides knowledge recall for the recognized entities and errors, and supplementation is provided for follow-up repair; and 4, correcting errors in a text result of the plain text data by an error repairing module through the acquired context information and supplementary data provided by the database, and outputting a document.

Description

Technical Field

[0001] The present invention relates to the technical field of text processing, and in particular to a complex text OCR error recognition and repair method based on a large language model. Background Art

[0002] With the rapid development of information technology, the document processing of enterprises and organizations has gradually moved towards electronicization, which has not only improved the efficiency of information management, but also improved the convenience of file sharing. However, due to the various formats and styles involved in the files, traditional text processing technology has shown obvious shortcomings when dealing with non-standard format files (such as scanned copies with more manual intervention such as stamps and signatures). Although traditional OCR (optical character recognition) technology has achieved certain results in text recognition of ordinary documents, it often faces many challenges in the parsing process of complex documents, including seal interference, overlapping signature positions, low-resolution text blur, and loss of structured content. These factors lead to limitations in the accuracy and completeness of the text results recognized by OCR, making it difficult to directly use the parsing results for subsequent information management and data analysis, resulting in a bottleneck in information processing efficiency.

[0003] At present, some new technical solutions attempt to apply deep learning to the field of text recognition and document parsing, trying to improve the recognition effect through the contextual understanding ability of the model, which has improved the accuracy of OCR technology to a certain extent. However, most of these methods are based on the training of a large amount of annotated data to achieve a high accuracy rate, and some complex text data are highly sensitive and confidential, resulting in slow and costly annotation work. In addition, deep learning models often focus more on the recognition of text content and ignore key auxiliary information in the file (such as watermarks, annotations, etc.), which makes the parsing results still lack a comprehensive restoration of the original document. It can be seen that the existing technology cannot meet the needs of electronic processing of non-standard format files, especially in the accurate identification of key fields and the guarantee of content integrity.

[0004] Therefore, technicians in this field are committed to developing a new complex text OCR error recognition and repair method to solve the above-mentioned defects in the prior art. Summary of the invention

[0005] In view of the above-mentioned defects of the prior art, the technical problem to be solved by the present invention is how to solve the errors and accuracy problems existing in OCR recognition. In order to better deal with the above-mentioned problems and improve the OCR recognition effect of complex texts, the present invention provides a complex text OCR error recognition and repair method based on a large language model, which can not only identify and correct errors in OCR recognition, but also perform detailed processing on key information such as names, place names, dates, etc., to ensure the accuracy and consistency of the recognition results.

[0006] Specifically, the present invention provides a complex text OCR error recognition and repair method based on a large language model, the method comprising the following steps: Step 1: The text preprocessing module identifies and excludes non-text areas in the preliminary text results generated by OCR to ensure that the remaining content is pure text input and obtain a text result of pure text data; Step 2: The error recognition module uses a large language model to perform entity recognition on the text results of the plain text data, and provides revision suggestions for the suspected errors identified; Step 3: The database provides knowledge recall for the identified entities and errors, supplementing the subsequent repairs; Step 4: The error repair module corrects the errors in the text result of the plain text data by using the acquired context information and the supplementary data provided by the database, and outputs the document.

[0007] Furthermore, the step 1 also pre-processes the obtained text result of the plain text data, including denoising or formatting.

[0008] Furthermore, the preprocessing also includes paragraph segmentation.

[0009] Furthermore, the step 2 includes the following sub-steps: Step 2.1, the error recognition module uses a large language model to perform entity recognition and identify key fields in the text result of the plain text data; Step 2.2: Based on the identified document content, the error identification module performs logic verification, marks suspected errors and provides modification suggestions.

[0010] Furthermore, the key fields in step 2.1 include names of people, places or dates.

[0011] Furthermore, the database in step 3 is an internal knowledge base.

[0012] Furthermore, the database in step 3 may also be an external database.

[0013] Furthermore, the error repair module in step 4 uses a large language model to correct errors.

[0014] Furthermore, step 4 includes the following sub-steps: Step 4.1, embedding the knowledge recall provided by the database into the input of the large language model as additional context; Step 4.2: The large language model combines the acquired context information and the knowledge recall to determine whether the revision suggestions provided by the error recognition module are feasible, and makes corrections based on the judgment result and outputs the document.

[0015] The complex text OCR error recognition and repair method based on a large language model provided by the present invention has at least the following technical effects: 1. The technical solution proposed in the present invention performs preliminary cleaning and filtering on the scanned files in the text preprocessing stage, so that the system can better remove non-text information such as seals and signatures to ensure the clarity and purity of the input content. This process provides more accurate input for the subsequent large language model repair module, improving the model's repair effect and text integrity; 2. The technical solution proposed in the present invention uses the contextual understanding ability of the large language model to automatically identify key entities (such as names, place names, dates, etc.) in complex texts and mark contextual errors, thereby achieving accurate repair based on error recognition. Compared with the traditional method of directly outputting OCR recognition results, the technical solution provided by the present invention introduces external database recall and contextual semantic analysis, which can incorporate more background information into error repair, making the recognition and repair results more consistent and accurate.

[0016] In summary, the technical solution provided by the present invention has significant advantages in the accuracy of OCR recognition and repair of complex texts, effectively making up for the shortcomings of existing OCR technology in non-standard format files, not only improving the recognition accuracy, content integrity and management efficiency of the text, but also being widely applicable to the electronic processing of various complex texts, laying a solid technical foundation for the efficient management and long-term utilization of texts.

[0017] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic diagram of a method flow of a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following describes several preferred embodiments of the present invention with reference to the drawings in the specification, so that the technical content is clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the protection scope of the present invention is not limited to the embodiments mentioned in the text.

[0020] Traditional OCR recognition technology usually shows low recognition accuracy when processing non-standard format files, such as stamped scans or files containing handwritten signatures, which can easily lead to parsing failures or a large number of recognition errors. Since OCR technology has difficulty accurately extracting text information when images are complex and text is ambiguous, these errors directly affect the readability of the file and the accuracy of subsequent analysis. In addition, the text generated by OCR often lacks the structured layout of the original file, which further reduces the integrity and readability of the file, bringing considerable difficulties to the archiving and retrieval of the file.

[0021] In response to the above-mentioned errors and accuracy problems in OCR recognition, an embodiment of the present invention proposes a complex text OCR error recognition and repair method based on a large language model, combining OCR recognition technology with the contextual understanding ability of a large language model, aiming to improve the accuracy and efficiency of electronic processing of complex texts, while ensuring the readability of the file content and the integrity of the structure. Through the above-mentioned innovative technical path, errors in documents can be efficiently identified and accurately repaired based on contextual information, which is particularly suitable for complex documents in non-standard formats such as seals and signatures, providing reliable technical support for document management of enterprises and organizations. By introducing the contextual understanding ability of a large language model, the embodiment of the present invention not only effectively improves the accuracy of OCR recognition, but also significantly improves the processing ability of unstructured content in documents, especially in terms of retaining and identifying key information.

[0022] Example 1 like Figure 1 As shown, an embodiment of the present invention provides a complex text OCR error recognition and repair method based on a large language model, comprising the following steps: Step 1: The text preprocessing module receives the preliminary text results generated by OCR, performs a series of filtering and cleaning operations, identifies and excludes non-text areas in the preliminary text results, so as to ensure that the remaining content is pure text input, and obtains text results of pure text data; wherein, the non-text areas usually include seals, signatures and other non-text components, which will interfere with the accuracy of subsequent text processing; Step 2: The error recognition module uses a large language model to perform entity recognition on the text results of the plain text data and provides revision suggestions for the suspected errors identified; Step 3: The database provides knowledge recall for the identified entities and errors, and provides supplement for subsequent repairs; wherein the database is an internal knowledge base or an external database; Step 4: The error repair module corrects the errors in the text results of the plain text data by using the context information obtained and the supplementary data provided by the database, and outputs the document.

[0023] In particular, step 1 also preprocesses the text results of the obtained plain text data to improve the data quality and lay the foundation for the error identification and repair steps; In particular, preprocessing includes denoising or formatting; In particular, preprocessing also includes paragraph segmentation.

[0024] Example 2 Based on Example 1, the plain text data after text preprocessing will be passed to the error recognition module. This module uses the context analysis capability of the large language model to automatically recognize key entities in complex texts and provide revision suggestions for possible erroneous OCR results.

[0025] Step 2 includes the following sub-steps: Step 2.1, the error recognition module uses the large language model to perform entity recognition and identify the key fields in the text results of the plain text data; Step 2.2: Based on the identified document content, the error recognition module performs logical verification, marks suspected errors and provides modification suggestions. For example, for names of people and places that may be misidentified, the error recognition module will search for possible spelling errors or content that does not match the context. The suspected errors identified will be marked and provided with modification suggestions for further processing by the subsequent repair module.

[0026] In particular, the key fields in step 2.1 include names of people, places or dates, which are the core content of the file.

[0027] Example 3 Based on Example 1 or 2, the context-based error repair module will correct suspected errors in the text. This module provides accurate repair support for the large language model through the acquired context information and database supplementary data.

[0028] Specifically, step 4 includes the following sub-steps: Step 4.1: Embed the knowledge recall provided by the database as additional context into the input of the large language model; Step 4.2: The large language model combines the acquired contextual information and knowledge recall to determine whether the revision suggestions provided by the error recognition module are feasible, and makes corrections based on the judgment results and outputs the document. This context-based repair process ensures that the generated text content is highly consistent with the original text semantics and is accurate.

[0029] Through the mutual collaboration of the above modules, the embodiment of the present invention provides an efficient and accurate complex text OCR error recognition and repair process, which is particularly suitable for texts with non-standard formats such as seals and signatures. It can not only accurately correct OCR text recognition errors, but also ensure the semantic consistency and information integrity of the output files, providing comprehensive technical support for the digitization of various documents and information management.

[0030] The preferred specific embodiments of the present invention are described in detail above. It should be understood that ordinary technicians in the field can make many modifications and changes based on the concept of the present invention without creative work. Therefore, all technical solutions that can be obtained by technicians in the technical field based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A complex text OCR error recognition and repair method based on a large language model, characterized in that: The method comprises the following steps: Step 1: The text preprocessing module identifies and excludes non-text areas in the preliminary text results generated by OCR to ensure that the remaining content is pure text input and obtain a text result of pure text data; Step 2: The error recognition module uses a large language model to perform entity recognition on the text results of the plain text data, and provides revision suggestions for the suspected errors identified; Step 3: The database provides knowledge recall for the identified entities and errors, supplementing the subsequent repairs; Step 4: The error repair module corrects the errors in the text result of the plain text data by using the acquired context information and the supplementary data provided by the database, and outputs the document.

2. The complex text OCR error recognition and repair method based on a large language model as claimed in claim 1, characterized in that: The step 1 also pre-processes the obtained text result of the plain text data, including denoising or formatting.

3. According to the complex text OCR error recognition and repair method based on a large language model as described in claim 2, the preprocessing also includes paragraph segmentation.

4. The complex text OCR error recognition and repair method based on a large language model as claimed in claim 1, characterized in that: The step 2 includes the following sub-steps: Step 2.1, the error recognition module uses a large language model to perform entity recognition and identify key fields in the text result of the plain text data; Step 2.2: Based on the identified document content, the error identification module performs logic verification, marks suspected errors and provides modification suggestions.

5. The complex text OCR error recognition and repair method based on a large language model as claimed in claim 4, characterized in that: The key fields in step 2.1 include names of people, places or dates.

6. The complex text OCR error recognition and repair method based on a large language model as claimed in claim 1, characterized in that: The database in step 3 is an internal knowledge base.

7. According to the complex text OCR error recognition and repair method based on a large language model as described in claim 6, the database in step 3 can also be an external database.

8. The complex text OCR error recognition and repair method based on a large language model as claimed in claim 1, characterized in that: The error repair module in step 4 uses a large language model to correct errors.

9. The complex text OCR error recognition and repair method based on a large language model as claimed in claim 8, characterized in that: The step 4 includes the following sub-steps: Step 4.1, embedding the knowledge recall provided by the database into the input of the large language model as additional context; Step 4.2: The large language model combines the acquired context information and the knowledge recall to determine whether the revision suggestions provided by the error recognition module are feasible, and makes corrections based on the judgment result and outputs the document.

Images