A method, system, device and storage medium for enhancing OCR recognition robustness
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA ASEAN INFORMATION PORT CO LTD
- Filing Date
- 2022-12-30
- Publication Date
- 2026-07-21
Smart Images
Figure CN116030475B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of OCR recognition technology, and more specifically, to a method, system, device, and storage medium for enhancing the robustness of OCR recognition. Background Technology
[0002] With the development of computing hardware and the popularization of big data, artificial intelligence-related technologies are finding increasing applications in people's daily lives. Computer vision, one of the four major areas of artificial intelligence, has gained recognition in recent years. OCR (Optical Character Recognition) technology within computer vision has been implemented in various real-world scenarios. Its convenience, speed, accuracy, and wide coverage provide intelligent assistance for the rapid conversion of unstructured data into structured data. Compared to the previous manual methods of quickly converting unstructured data into structured data, it offers advantages such as faster speed, higher accuracy, and significant savings in human resources. It can quickly achieve text extraction in various application scenarios, including images, invoices, documents, and certificates.
[0003] However, existing OCR recognition models are prone to misjudgment and missed judgment when recognizing complex real-life image data. Summary of the Invention
[0004] The technical problem to be solved by the present invention is to address the above-mentioned shortcomings of the prior art. The purpose of the present invention is to provide a method to enhance the robustness of OCR recognition.
[0005] The second objective of this invention is to provide a system that enhances the robustness of OCR recognition.
[0006] The third objective of this invention is to provide a computer device.
[0007] The fourth objective of this invention is to provide a computer-readable storage medium.
[0008] To achieve the first objective mentioned above, the present invention provides a method for enhancing the robustness of OCR recognition, comprising the following steps:
[0009] Step S1. Obtain the original training dataset and original model for OCR model training;
[0010] Step S2. Set the data augmentation coefficient and divide the data augmentation into multiple levels;
[0011] Step S3. Randomly pollute the original training dataset in Step S1 to obtain a polluted dataset. Data pollution includes at least one of five categories: color transformation, adding noise, blurring, deformation, and adding weather effects. At least one data pollution category is randomly selected for each image in the dataset, and the corresponding enhancement coefficient range is configured according to the data pollution category.
[0012] Step S4. Perform PGD adversarial attack on the original training dataset and original model from step S1 to obtain an adversarial example dataset;
[0013] Step S5. Cartoonize the original training dataset from step S1 to obtain a cartoonized dataset;
[0014] Step S6. Merge the original training dataset from Step S1, the polluted dataset from Step S3, the adversarial example dataset from Step S4, and the cartoonized dataset from Step S5 to obtain a data-centric augmented dataset as the final dataset.
[0015] Step S7. Retrain the original model using the final dataset from step S6 to obtain an enhanced robust OCR model.
[0016] As a further improvement, in step S2, data augmentation is divided into 5 levels, represented by coefficients 1 to 5, where 1 represents the weakest impact on the original image and 5 represents the strongest impact on the original image.
[0017] Further, in step S3,
[0018] Color transformation includes at least one of the following methods: Brightness, CLAHE, Contrast, Grayscale, and Saturation.
[0019] Adding noise includes at least one of GaussianNoise, ImpulseNoise, LaplaceNoise, PoissonNoise, SaltAndPepper, ShotNoise, and SpeckleNoise;
[0020] Blur processing includes at least one of the following methods: DefocusBlur, GaussianBlur, GlassBlur, MediaBlur, MotionBlur, Spatter, and ZoomBlur.
[0021] Deformation includes at least one of Superpixels, JPEG_Compression, Pixelation, and ElasticTransform;
[0022] Adding weather effects includes at least one of Snowflakes, Snow, Frost, Rain, Fog, Clouds, and FastSnowyLandscape.
[0023] Furthermore, in step S3, the pollution rule is specifically as follows: for each image, one or two pollution categories are randomly selected, and one method is randomly selected from each pollution category;
[0024] The enhancement coefficient for color transformation is randomly selected and ranges from coefficient 1 to coefficient 5;
[0025] The enhancement factor for the noise is randomly selected and ranges from 2 to 5.
[0026] The enhancement coefficients randomly selected in the fuzzy processing range from coefficient 2 to coefficient 4;
[0027] The range of enhancement coefficients for random deformation selection is from coefficient 1 to coefficient 3;
[0028] The enhancement coefficient for increasing weather effects is randomly selected and ranges from coefficient 1 to coefficient 3.
[0029] Furthermore, the contaminated dataset obtained in step S3 is repeatedly subjected to random contamination to obtain a new contaminated dataset;
[0030] In step S6, the original training dataset from step S1, the polluted dataset from step S3, the adversarial example dataset from step S4, the cartoonized dataset from step S5, and the new polluted dataset are merged to obtain a data-centric augmented dataset as the final dataset.
[0031] Furthermore, in step S4, the enhancement coefficient ranges from coefficient 2 to coefficient 5, and a coefficient is randomly selected from the enhancement coefficient range when generating adversarial examples for each data point;
[0032] In step S5, the enhancement coefficient ranges from coefficient 1 to coefficient 5, and a coefficient is randomly selected from the enhancement coefficient range during each data cartoonization process.
[0033] Furthermore, in step S7, during each data input during training, an affine transformation is performed on the image data with a probability of 30%, and a label smoothing parameter of 0.1 is added to the classification label. After training convergence, a robust OCR model is obtained.
[0034] To achieve the second objective mentioned above, the present invention provides a system for enhancing the robustness of OCR recognition, comprising:
[0035] The acquisition module is used to acquire the original training dataset and the original model during OCR model training.
[0036] The augmentation coefficient module is used to set the data augmentation coefficient, dividing data augmentation into multiple levels;
[0037] The pollution module is used to randomly pollute the original training dataset to obtain a polluted dataset.
[0038] The adversarial module is used to perform PGD adversarial attacks on the original training dataset and the original model to obtain an adversarial sample dataset.
[0039] The cartoonization module is used to cartoonize the original training dataset to obtain a cartoonized dataset;
[0040] The merging module is used to merge the original training dataset, the polluted dataset, the adversarial example dataset, and the cartoonized dataset to obtain a data-centric augmented dataset as the final dataset.
[0041] The training module is used to retrain the original model using the final dataset to obtain an enhanced robust OCR model.
[0042] To achieve the third objective mentioned above, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the aforementioned method for enhancing the robustness of OCR recognition.
[0043] To achieve the fourth objective mentioned above, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the aforementioned method for enhancing the robustness of OCR recognition.
[0044] Beneficial effects
[0045] Compared with the prior art, the present invention has the following advantages:
[0046] Based on a data-centric approach, this invention randomly combines five image contamination strategies and one image cartoonization strategy, integrates adversarial examples, expands the original training data to five times its original size, and retrains the model. Without changing the model structure, a more robust OCR model is obtained. Attached Figure Description
[0047] Fig. 1 This is a flowchart of the enhancement method in this invention;
[0048] Fig. 2 This is a list diagram illustrating image contamination enhancement in this invention. Detailed Implementation
[0049] The present invention will be further described below with reference to specific embodiments shown in the accompanying drawings.
[0050] See Figs. 1-2 A method to enhance the robustness of OCR recognition includes the following steps:
[0051] Step S1. Obtain the original training dataset and the original model for OCR model training. The original training dataset is the initial dataset without any data augmentation or augmentation operations, and the original model is the OCR model that has converged after training with the original training dataset.
[0052] Step S2. Set the data augmentation coefficient and divide the data augmentation into multiple levels. There are 5 levels of data augmentation, represented by coefficients 1 to 5, where 1 represents the weakest impact on the original image and 5 represents the strongest impact on the original image.
[0053] Step S3. Randomly corrupt the original training dataset from Step S1 to obtain a corrupted dataset. Data corruption includes at least one of five categories: color transformation, adding noise, blurring, deformation, and adding weather effects. At least one data corruption category is randomly selected for each image in the dataset, and the corresponding enhancement coefficient range is configured according to the data corruption category.
[0054] Color transformation includes at least one of the following methods: Brightness, CLAHE (histogram equalization), Contrast, Grayscale, and Saturation.
[0055] Adding noise includes at least one of Gaussian noise, impulse noise, Laplace noise, Poisson noise, Salt and Pepper noise, Shot noise, and speckle noise.
[0056] Blur processing includes at least one of the following methods: DefocusBlur, GaussianBlur, GlassBlur, MediaBlur, MotionBlur, Spatter, and ZoomBlur.
[0057] The deformation includes at least one of the following methods: Superpixels, JPEG Compression, Pixelation, and Elastic Transform.
[0058] Adding weather effects includes at least one of the following methods: Snowflakes, Snow, Frost, Rain, Fog, Clouds, and FastSnowyLandscape.
[0059] The specific contamination rules are as follows: for each image, one or two major contamination categories are randomly selected, and one method is randomly selected from each major contamination category.
[0060] The enhancement coefficient for color transformation is randomly selected and ranges from coefficient 1 to coefficient 5.
[0061] The enhancement coefficient for the noise is randomly selected and ranges from coefficient 2 to coefficient 5.
[0062] The enhancement coefficients randomly selected for fuzzy processing range from coefficient 2 to coefficient 4.
[0063] The range of enhancement coefficients for random deformation selection is from coefficient 1 to coefficient 3.
[0064] The enhancement coefficient for increasing weather effects is randomly selected and ranges from coefficient 1 to coefficient 3.
[0065] Step S4. Perform PGD adversarial attacks on the original training dataset and original model from Step S1 to obtain an adversarial example dataset. The enhancement coefficients range from coefficient 2 to coefficient 5, and a coefficient is randomly selected from this range when generating adversarial examples for each dataset.
[0066] Step S5. Cartoonize the original training dataset from Step S1 to obtain a cartoonized dataset. The augmentation coefficients range from coefficient 1 to coefficient 5, and a coefficient is randomly selected from this range for each dataset during the cartoonization process.
[0067] Step S6. Merge the original training dataset from Step S1, the polluted dataset from Step S3, the adversarial example dataset from Step S4, and the cartoonized dataset from Step S5 to obtain a data-centric augmented dataset as the final dataset, i.e., a completely new dataset that is 4 times larger than the original training data as the final dataset.
[0068] Step S7. Retrain the original model using the final dataset from Step S6 to obtain an enhanced robust OCR model. Specifically, during training, for each data input, perform an affine transformation on the image data with a 30% probability, add label smoothing with a parameter of 0.1 to the classification labels, and obtain a robust enhanced OCR model after training convergence.
[0069] Preferably, the contaminated dataset obtained in step S3 is repeatedly subjected to random contamination, with the data contamination class and contamination rules being the same as in step S3, resulting in a new contaminated dataset. In step S6, the original training dataset in step S1, the contaminated dataset in step S3, the adversarial example dataset in step S4, the cartoonized dataset in step S5, and the new contaminated dataset are merged to obtain a data-centric augmented dataset as the final dataset, i.e., a completely new dataset that is 5 times larger than the original training data is obtained as the final dataset. The dataset is expanded, and the trained OCR model is more robust.
[0070] A system for enhancing the robustness of OCR recognition includes:
[0071] The acquisition module is used to acquire the original training dataset and the original model during OCR model training.
[0072] The augmentation coefficient module is used to set the data augmentation coefficient, dividing data augmentation into multiple levels;
[0073] The pollution module is used to randomly pollute the original training dataset to obtain a polluted dataset.
[0074] The adversarial module is used to perform PGD adversarial attacks on the original training dataset and the original model to obtain an adversarial sample dataset.
[0075] The cartoonization module is used to cartoonize the original training dataset to obtain a cartoonized dataset;
[0076] The merging module is used to merge the original training dataset, the polluted dataset, the adversarial example dataset, and the cartoonized dataset to obtain a data-centric augmented dataset as the final dataset.
[0077] The training module is used to retrain the original model using the final dataset to obtain an enhanced robust OCR model.
[0078] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the above-described method for enhancing the robustness of OCR recognition.
[0079] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method for enhancing the robustness of OCR recognition.
[0080] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention, and these will not affect the effectiveness of the implementation of the present invention or the practicality of the patent.
Claims
1. A method for enhancing the robustness of OCR recognition, characterized in that, Includes the following steps: Step S1. Obtain the original training dataset and original model for OCR model training; Step S2. Set the data augmentation coefficient and divide the data augmentation into multiple levels; the data augmentation is divided into 5 levels, represented by coefficients 1 to 5, where 1 represents the weakest impact on the original image and 5 represents the strongest impact on the original image; Step S3. Randomly pollute the original training dataset in Step S1 to obtain a polluted dataset. Data pollution includes at least one of five categories: color transformation, adding noise, blurring, deformation, and adding weather effects. At least one data pollution category is randomly selected for each image in the dataset, and the corresponding enhancement coefficient range is configured according to the data pollution category. Step S4. Perform PGD adversarial attack on the original training dataset and original model from step S1 to obtain an adversarial example dataset; Step S5. Cartoonize the original training dataset from step S1 to obtain a cartoonized dataset; Step S6. Merge the original training dataset from Step S1, the polluted dataset from Step S3, the adversarial example dataset from Step S4, and the cartoonized dataset from Step S5 to obtain a data-centric augmented dataset as the final dataset. Step S7. Retrain the original model using the final dataset from step S6 to obtain an enhanced robust OCR model.
2. The method for enhancing the robustness of OCR recognition according to claim 1, characterized in that, In step S3, Color transformation includes at least one of the following methods: Brightness, CLAHE, Contrast, Grayscale, and Saturation. Adding noise includes at least one of GaussianNoise, ImpulseNoise, LaplaceNoise, PoissonNoise, SaltAndPepper, ShotNoise, and SpeckleNoise; Blur processing includes at least one of the following methods: DefocusBlur, GaussianBlur, GlassBlur, MediaBlur, MotionBlur, Spatter, and ZoomBlur. Deformation includes at least one of Superpixels, JPEG_Compression, Pixelation, and ElasticTransform; Adding weather effects includes at least one of Snowflakes, Snow, Frost, Rain, Fog, Clouds, and FastSnowyLandscape.
3. The method for enhancing the robustness of OCR recognition according to claim 2, characterized in that, In step S3, the specific contamination rule is that one or two contamination categories are randomly selected for each image, and one method is randomly selected from each contamination category. The enhancement coefficient for color transformation is randomly selected and ranges from coefficient 1 to coefficient 5; The enhancement factor for the noise is randomly selected and ranges from 2 to 5. The enhancement coefficients randomly selected in the fuzzy processing range from coefficient 2 to coefficient 4; The range of enhancement coefficients for random deformation selection is from coefficient 1 to coefficient 3; The enhancement coefficient for increasing weather effects is randomly selected and ranges from coefficient 1 to coefficient 3.
4. The method for enhancing the robustness of OCR recognition according to claim 1, characterized in that, The polluted dataset obtained in step S3 is repeatedly subjected to random pollution to obtain a new polluted dataset. In step S6, the original training dataset from step S1, the polluted dataset from step S3, the adversarial example dataset from step S4, the cartoonized dataset from step S5, and the new polluted dataset are merged to obtain a data-centric augmented dataset as the final dataset.
5. The method for enhancing the robustness of OCR recognition according to claim 1, characterized in that, In step S4, the enhancement coefficient ranges from coefficient 2 to coefficient 5, and a coefficient is randomly selected from the enhancement coefficient range when generating adversarial examples for each data point; In step S5, the enhancement coefficient ranges from coefficient 1 to coefficient 5, and a coefficient is randomly selected from the enhancement coefficient range during each data cartoonization process.
6. The method for enhancing the robustness of OCR recognition according to claim 1, characterized in that, In step S7, during each data input during training, an affine transformation is performed on the image data with a probability of 30%, and a label smoothing parameter of 0.1 is added to the classification label. After training convergence, a robust OCR model is obtained.
7. A system for enhancing the robustness of OCR recognition, characterized in that, include: The acquisition module is used to acquire the original training dataset and the original model during OCR model training. The augmentation coefficient module is used to set the data augmentation coefficient, dividing data augmentation into multiple levels; Data augmentation is divided into 5 levels, represented by coefficients 1 to 5, where 1 represents the weakest impact on the original image and 5 represents the strongest impact on the original image. The contamination module is used to randomly contaminate the original training dataset to obtain a contaminated dataset. Data contamination includes at least one of five categories: color transformation, noise addition, blurring, deformation, and weather effect addition. At least one data contamination category is randomly selected for each image in the dataset, and the corresponding enhancement coefficient range is configured according to the data contamination category. The adversarial module is used to perform PGD adversarial attacks on the original training dataset and the original model to obtain an adversarial sample dataset. The cartoonization module is used to cartoonize the original training dataset to obtain a cartoonized dataset; The merging module is used to merge the original training dataset, the polluted dataset, the adversarial example dataset, and the cartoonized dataset to obtain a data-centric augmented dataset as the final dataset. The training module is used to retrain the original model using the final dataset to obtain an enhanced robust OCR model.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements a method for enhancing the robustness of OCR recognition as described in any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a method for enhancing the robustness of OCR recognition as described in any one of claims 1 to 6.