OCR (Optical Character Recognition) error correction method, system and equipment based on large model and medium
Through the OCR recognition error correction method based on large models, the deep learning model is used to correct the OCR recognition results and optimize the model, which solves the problem of decreasing the accuracy of noun recognition in special industries and improves the recognition accuracy and efficiency.
Patent Information
- Application Number
- CN202410989497.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-19
AI Technical Summary
The existing OCR recognition technology has problems such as decreasing recognition accuracy and inaccurate correction when dealing with proprietary nouns in special industries, especially in areas such as drug names.
Using the OCR recognition and error correction method based on large models, through data preprocessing, text content extraction, directional fine-tuning training, OCR error correction and error correction feedback steps, the deep learning model is used to correct the OCR recognition results, and directional fine-tuning is used to correct the OCR recognition results, and directional fine-tuning is used to correct the OCR recognition results. Optimize the model to improve identification accuracy.
It improves the accuracy and efficiency of OCR recognition, solves the problem of inaccurate correction of proper noun recognition in special industries, and has high application value.
Smart Images

Figure CN118982835A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of OCR recognition and large-model text error correction in specific industries, and in particular to a large-model-based OCR recognition error correction method, system, device and medium. Background Art
[0002] At present, OCR recognition technology is widely used in the manufacturing industry. However, some special industries may have product-specific nouns or easily confused nouns with fewer samples, which will lead to a decrease in OCR recognition accuracy. For example, OCR recognition may not accurately correct special industry-specific nouns such as drug names. Summary of the invention
[0003] The object of the present invention is to provide an OCR recognition error correction method, system, device and medium based on a large model, thereby solving all or one of the above-mentioned problems existing in the prior art.
[0004] In order to solve the above technical problems, the specific technical solutions of the present invention are as follows: On the one hand, the present invention provides an OCR recognition and error correction method based on a large model, comprising the following steps: Data preprocessing: Perform image preprocessing on training images; Text content extraction: The preprocessed image is input into the large model to be trained for feature extraction and text recognition; Directed fine-tuning training: Adjusting parameters and optimizing the model of the large model to be trained based on the text recognition content; OCR Correction: Input the OCR recognition results into the trained large model for error correction; Error correction feedback: According to the error correction result of the trained large model on the OCR recognition result, the performance of the trained large model is evaluated.
[0005] As an improved solution, the data preprocessing step further includes: Denoising the training image by using mean filtering, median filtering and Gaussian filtering; Converting the training image into black and white; The tilt in the training image is corrected.
[0006] As an improved solution, the text content extraction step further includes: Using the large model to be trained to perform feature extraction on the preprocessed image, extracting features of text information and its context information from the preprocessed image; Based on the extracted image features, text recognition is performed using a deep learning model.
[0007] As an improved solution, the directional fine-tuning training step further includes: Taking the extracted industry text content as input, performing targeted fine-tuning on the large model to be trained; Adjust hyperparameters and model structure by monitoring changes in loss functions and evaluation indicators during training; The validation set is used to evaluate the fine-tuned large model to be trained, and the model structure, parameters and learning rate of the large model to be trained are optimized according to the evaluation results.
[0008] As an improved solution, the OCR error correction step further includes: Input the OCR recognition results into the fine-tuned trained large model; The model performs error correction on the OCR recognition results based on the input content and training results; The model outputs error correction suggestions and corresponding thresholds; Make a business judgment based on the error correction suggestions and corresponding thresholds output by the model, and choose to accept the error correction suggestions or ignore them.
[0009] As an improved solution, the error correction feedback step further includes: The images and contents with incorrect correction in the correction results are input into the trained large model for large model fine-tuning training.
[0010] As an improved solution, the error correction feedback step further includes: The trained model is evaluated using a new test data set, and based on the evaluation result, it is determined whether to continue collecting new error-corrected images and content for the next round of fine-tuning training.
[0011] On the other hand, the present invention also provides an OCR recognition and error correction system based on a large model, comprising: The data preprocessing module is used to: perform image preprocessing on training images; The text content extraction module is used to: input the preprocessed image into the large model to be trained for feature extraction and text recognition; A directional fine-tuning training module, used to: adjust parameters and optimize the model of the large model to be trained based on the text recognition content; The OCR error correction module is used to: input the OCR recognition results into the trained large model for error correction; The error correction feedback module is used to: perform performance evaluation on the trained large model according to the error correction result of the trained large model on the OCR recognition result.
[0012] On the other hand, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the OCR recognition and error correction method based on a large model are implemented.
[0013] On the other hand, the present invention further provides a computer device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; wherein: The memory is used to store computer programs; The processor is used to execute the steps of the large model-based OCR recognition and error correction method by running the program stored in the memory.
[0014] The beneficial effects of the technical solution of the present invention are: 1. The large model-based OCR recognition and error correction method described in the present invention can utilize the powerful capabilities of the large model in the field of text understanding. By using the directional fine-tuning training of the standard error correction large model on the OCR recognition results of different industries, the problem of inaccurate recognition and error correction of special industry proper nouns such as drug names in OCR recognition is solved, the accuracy and efficiency of OCR recognition are improved, the defects of the existing technology are compensated, and it has a high application value.
[0015] 2. The large model-based OCR recognition and error correction system described in the present invention can realize the large model-based OCR recognition and error correction method described in the present invention through the mutual cooperation of system modules.
[0016] 3. The computer-readable storage medium described in the present invention can guide the system modules to cooperate, thereby realizing the large model-based OCR recognition and error correction method described in the present invention, and the computer-readable storage medium described in the present invention also effectively improves the operability of the large model-based OCR recognition and error correction method.
[0017] 4. The computer device described in the present invention can store and execute the computer-readable storage medium, thereby realizing the large model-based OCR recognition and error correction method described in the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0019] Figure 1 It is a flowchart of the OCR recognition and error correction method based on a large model described in Example 1 of the present invention; Figure 2 It is a logical flow diagram of the large model-based OCR recognition and error correction method described in Example 1 of the present invention; Figure 3 Schematic diagram of the architecture of the large model-based OCR recognition and error correction system described in Example 2 of the present invention; Figure 4 is a schematic diagram of the structure of the computer device described in Example 4 of the present invention; The symbols in the accompanying drawings are explained as follows: 1501, processor; 1502, communication interface; 1503, memory; 1504, communication bus. DETAILED DESCRIPTION
[0020] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.
[0021] In the description of the present invention, it should be noted that the embodiments described in the present invention are only part of the embodiments of the present invention, rather than all of the embodiments; based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.
[0022] The terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this article described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment. Example 1
[0023] This embodiment provides an OCR recognition and error correction method based on a large model, such as Figure 1 and Figure 2 As shown, the following steps are included: The core function of this method is to utilize the powerful ability of the big model in the field of text understanding, and to solve the problem of inaccurate OCR recognition of special industry proper nouns such as drug names by fine-tuning the OCR recognition results of different industries through the standard error correction big model, thereby improving the accuracy and efficiency of OCR recognition. The details are as follows: S100, data preprocessing: Specifically, image denoising: reduce the noise in the image and improve the accuracy of OCR recognition through mean filtering, median filtering, Gaussian filtering, etc. Specifically, image binarization: converting the image into black and white to simplify image information and improve OCR recognition efficiency; Specifically, image tilt correction: correct the tilt in the OCR image (i.e., training image) to ensure that the text lines are arranged horizontally and improve the accuracy of OCR recognition.
[0024] S200, text content extraction: Specifically, feature extraction: use a dedicated large model to extract features from images, extracting features of text information and its contextual information from preprocessed images; Specifically, text recognition: Based on the extracted image features, text recognition is performed using a large deep learning model.
[0025] S300, directional fine-tuning training: Specifically, targeted fine-tuning: taking the extracted industry-specific text content as input, the pre-trained model is fine-tuned in a targeted manner; this is achieved by adding additional layers to the model or modifying existing layers to adapt to the OCR recognition task of a specific industry; Specifically, the training process: determine the learning rate, batch size, number of training rounds and other hyperparameters through experiments to obtain the best model performance and set appropriate hyperparameters; monitor the training process, including changes in loss functions and evaluation indicators, and adjust hyperparameters and model structures in a timely manner; Specifically, model evaluation and optimization: Use the validation set to evaluate the fine-tuned model to check its performance on industry-specific OCR recognition tasks; optimize the model based on the evaluation results, including adjusting the model structure, parameters, and learning rate.
[0026] S400, OCR error correction: Specifically, input the OCR recognition result: input the OCR recognition result into the fine-tuned trained large model; Specifically, error correction and verification: the model performs error correction and verification on the OCR recognition results based on the input content and training results; Specifically, output error correction suggestions and thresholds: The model outputs error correction suggestions and corresponding thresholds; this threshold is the model's confidence in the error correction suggestions; Specifically, business judgment: users can make business judgments based on the error correction suggestions and thresholds output by the model; if the threshold of the error correction suggestion is high, or the error correction suggestion is highly matched with the business scenario, the user can choose to accept the error correction suggestion; otherwise, they can choose to ignore or modify it manually.
[0027] S500, Error correction feedback: Specifically, error correction feedback processing: the images and content with error correction in a specific industry are input into the corresponding industry classification model for fine-tuning training of the model; Specifically, model evaluation and iteration: Use a new test dataset to evaluate the model after error correction return processing to check its performance and accuracy on error correction tasks in specific industries; if the model performance and accuracy meet expectations, the model can be deployed in practical applications; if the performance and accuracy are not good, continue to collect new error-corrected images and their contents for the next round of fine-tuning training.
[0028] It should be noted that the above examples are only for explaining the present invention and cannot limit the protection scope of the present invention. Example 2
[0029] This embodiment is based on the same inventive concept as the large model-based OCR recognition and error correction method described in Example 1, and provides a large model-based OCR recognition and error correction system, as shown in the figure, including: The data preprocessing module is used to: perform image preprocessing on training images; The text content extraction module is used to: input the preprocessed image into the large model to be trained for feature extraction and text recognition; A directional fine-tuning training module, used to: adjust parameters and optimize the model of the large model to be trained based on the text recognition content; The OCR error correction module is used to: input the OCR recognition results into the trained large model for error correction; The error correction feedback module is used to: perform performance evaluation on the trained large model according to the error correction result of the trained large model on the OCR recognition result. Example 3
[0030] This embodiment provides a computer-readable storage medium, including: The storage medium is used to store computer software instructions used to implement the large model-based OCR recognition and error correction method described in the above-mentioned Example 1, which includes instructions for executing the above-mentioned program set for the large model-based OCR recognition and error correction method; specifically, the executable program can be built into the large model-based OCR recognition and error correction system described in Example 2, so that the large model-based OCR recognition and error correction system can implement the large model-based OCR recognition and error correction method described in Example 1 by executing the built-in executable program.
[0031] In addition, the computer-readable storage medium of this embodiment may adopt any combination of one or more computer-readable storage media, wherein the computer-readable storage medium includes electrical, optical, electromagnetic, infrared or semiconductor systems, devices or components, or any combination thereof. Example 4
[0032] This embodiment provides an electronic device, such as Figure 4 As shown, the electronic device may include: a processor 1501 , a communication interface 1502 , a memory 1503 and a communication bus 1504 , wherein the processor 1501 , the communication interface 1502 , and the memory 1503 communicate with each other via the communication bus 1504 .
[0033] Memory 1503, used for storing computer programs; The processor 1501 is used to implement the steps of the large model-based OCR recognition and error correction method described in the above-mentioned embodiment 1 when executing the computer program stored in the memory 1503.
[0034] As an embodiment of the present invention, the communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0035] As an implementation mode of the present invention, the communication interface is used for communication between the above-mentioned terminal and other devices.
[0036] As an embodiment of the present invention, the memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0037] As an embodiment of the present invention, the above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
[0038] Different from the existing technology, the present application adopts a large model-based OCR recognition and error correction method, system, device and medium, which can utilize the powerful ability of the large model in the field of text understanding. By using the targeted fine-tuning training of the standard error correction large model for the OCR recognition results of different industries, the problem of inaccurate recognition and error correction of special industry proper nouns such as drug names is solved, the accuracy and efficiency of OCR recognition are improved, the defects of the existing technology are compensated, and it has a high application value.
[0039] It should be understood that in the various embodiments of this document, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this document.
[0040] It should also be understood that in the embodiments of this article, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0041] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this article.
[0042] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0043] In the several embodiments provided herein, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or can be electrical, mechanical or other forms of connection.
[0044] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of this article.
[0045] In addition, each functional unit in each embodiment of this invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional unit.
[0046] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this article is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of this article. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0047] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A large model-based OCR recognition and error correction method, characterized in that: The following steps are involved: Data preprocessing: Perform image preprocessing on training images; Text content extraction: The preprocessed image is input into the large model to be trained for feature extraction and text recognition; Directed fine-tuning training: Adjusting parameters and optimizing the model of the large model to be trained based on the text recognition content; OCR Correction: Input the OCR recognition results into the trained large model for error correction; Error correction feedback: According to the error correction result of the trained large model on the OCR recognition result, the performance of the trained large model is evaluated.
2. The OCR recognition and error correction method based on a large model according to claim 1, characterized in that: The data preprocessing step further comprises: Denoising the training image by using mean filtering, median filtering and Gaussian filtering; Converting the training image into black and white; The tilt in the training image is corrected.
3. The OCR recognition and error correction method based on a large model according to claim 1, characterized in that: The text content extraction step further comprises: Using the large model to be trained to perform feature extraction on the preprocessed image, extracting features of text information and its context information from the preprocessed image; Based on the extracted image features, text recognition is performed using a large deep learning model.
4. The OCR recognition and error correction method based on a large model according to claim 1, characterized in that: The directional fine-tuning training step further comprises: Taking the extracted industry text content as input, performing targeted fine-tuning on the large model to be trained; Adjust hyperparameters and model structure by monitoring changes in loss functions and evaluation indicators during training; The validation set is used to evaluate the fine-tuned large model to be trained, and the model structure, parameters and learning rate of the large model to be trained are optimized according to the evaluation results.
5. The OCR recognition and error correction method based on a large model according to claim 1, characterized in that: The OCR error correction step further comprises: Input the OCR recognition results into the fine-tuned trained large model; The model performs error correction on the OCR recognition results based on the input content and training results; The model outputs error correction suggestions and corresponding thresholds; Make a business judgment based on the error correction suggestions and corresponding thresholds output by the model, and choose to accept the error correction suggestions or ignore them.
6. The OCR recognition and error correction method based on a large model according to claim 1, characterized in that: The error correction feedback step further comprises: The images and contents with incorrect correction in the correction results are input into the trained large model for large model fine-tuning training.
7. The OCR recognition and error correction method based on a large model according to claim 1, characterized in that: The error correction feedback step further includes: The trained model is evaluated using a new test data set, and based on the evaluation result, it is determined whether to continue collecting new error-corrected images and content for the next round of fine-tuning training.
8. An OCR recognition and error correction system based on a large model, characterized in that: include: A data preprocessing module is used to: perform image preprocessing on training images; The text content extraction module is used to: input the preprocessed image into the large model to be trained for feature extraction and text recognition; A directional fine-tuning training module, used to: adjust parameters and optimize the model of the large model to be trained based on the text recognition content; The OCR error correction module is used to: input the OCR recognition results into the trained large model for error correction; The error correction feedback module is used to: perform performance evaluation on the trained large model according to the error correction result of the trained large model on the OCR recognition result.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the large model-based OCR recognition and error correction method described in any one of claims 1 to 7 are implemented.
10. A computer device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; wherein: The memory is used to store computer programs; The processor is used to execute the steps of the large model-based OCR recognition and error correction method described in any one of claims 1 to 7 by running the program stored in the memory.
Citation Information
Cited By
Document processing method and device and electronic equipment
CN120544208A
Unstructured document OCR error correction method and system based on multi-modal large model
CN121281073A
OCR (Optical Character Recognition) result correction method and system based on large model driving
CN121904782A
Bill identification method, apparatus and device, and computer program product
CN122200713A
