Product label printed text detection method and system based on PaddleOCR

Abstract

The present invention discloses a method and system for detecting printed text on product labels based on PaddleOCR, which relates to the field of automated quality inspection technology. The method comprises the following steps: uploading and storing template images and printed images corresponding to product labels, performing OCR content recognition on the template images and the printed images, and obtaining detection frames for each line of text; dynamically matching the detection frames in the template images and the printed images, and recording detection frames that are fully matched, partially matched, and unmatched; and visualizing the detection results of printed text on product labels by marking the detection frames with different preset colors based on the recognition and recording results. The present invention uses OCR to identify the template and printed content, dynamically calculates the similarity of the detection frames, and combines this with character-level difference positioning to provide intuitive visualization results, which is suitable for quality verification in the label printing industry.

Description

Technical Field

[0001] The present invention relates to the field of automated quality inspection technology, and more specifically, to a method and system for detecting printed text on product labels based on PaddleOCR. Background Art

[0002] With the rapid development of the product label printing industry, the variety and quantity of labels continues to increase. During the label production and printing process, factors such as production efficiency and process complexity can easily lead to problems such as label content deviations, misprints, and omissions. These errors not only affect product display but can also cause consumer confusion, damage brand image, and even lead to legal disputes.

[0003] Traditional label proofreading relies primarily on manual labor, where discrepancies are detected by manually comparing templates with printed labels. However, this approach presents the following challenges: High workload: Label content is complex and the quantity is large, making manual proofreading time-consuming and labor-intensive, making it difficult to meet the demands of large-scale production. Unstable accuracy: Manual proofreading is susceptible to factors such as fatigue and subjective judgment, resulting in a high error rate. Low efficiency: Given the high precision requirements, manual proofreading efficiency struggles to keep pace with production. Existing OCR technology in label proofreading scenarios suffers from issues such as line splitting, character matching failures, and inaccurate discrepancy location. PaddleOCR, an open-source OCR (optical character recognition) tool library developed by Baidu PaddlePaddle, supports multilingual text detection, recognition, and layout analysis, and features high accuracy, lightweight design, and easy deployment. To address the high workload and instability of traditional manual proofreading, there is an urgent need for an efficient and accurate proofreading method based on PaddleOCR technology. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention proposes a method and system for detecting printed text on product labels based on PaddleOCR. Through the strategies of adaptively merging detection frames, dynamic matching and similarity calculation, the method optimizes the problems of splitting a line of text and locating differences, greatly improving proofreading efficiency and accuracy, and reducing labor costs.

[0005] The present invention provides a method for detecting printed text on a product label based on PaddleOCR, comprising the following steps:

[0006] Upload and store the template image and printed image corresponding to the product label, pre-process the image, perform OCR content recognition on the template image and the printed image, and obtain the detection frame of each line of text;

[0007] Dynamically match the detection frames in the template image and the printed image, and record the detection frames that are fully matched, partially matched, and unmatched;

[0008] According to the recognition record results of the detection frame, different preset colors are used to mark them respectively to visualize the detection results of the printed text on the product label.

[0009] In this solution, template images and printed images corresponding to product labels are uploaded and stored, and the images are pre-processed as follows:

[0010] Obtain the corresponding template image according to the target product label, upload the template image and save it persistently in the database;

[0011] Obtain a printed image corresponding to the target product label, upload the printed image and store it in a temporary storage folder in the database, and clean up the temporary storage folder after the inspection task is completed;

[0012] The stored template images and printed images are subjected to grayscale processing, image noise reduction, image segmentation, tilt correction and binarization processing respectively to obtain the preprocessed printed images and construct the data set to be identified.

[0013] In this solution, PaddleOCR is used to perform content recognition from the template image and the printed image, specifically:

[0014] Call the PP-OCRv4 model in PaddleOCR to perform global OCR detection and recognition on the template image and the printed image. Use the DBNet pre-trained model to detect the text area in the image and output the detection box of the text area.

[0015] The text line image is cropped according to the detection frame, the CRNN pre-trained model is used for text content recognition, the PP-LCNet model is used to detect the text direction, the text direction is corrected for the text content recognition result, and the content of each line of text and the coordinate position of its detection frame are output.

[0016] In this solution, the detection frames in the template image and the printed image are dynamically matched, specifically:

[0017] Convert the recognized characters in the template image and the printed image into full-width and half-width characters, and perform nested loops on the converted template image OCR results and the printed image OCR results lists to achieve pairwise comparison;

[0018] According to the paired comparison results, the fully matched detection frame, partially matched detection frame and unmatched detection frame are recorded.

[0019] In this solution, adaptive merging is used to split the detection frame. Character-level similarity analysis and dynamic expansion of the detection frame are used to locate text differences. Specifically:

[0020] The average pixel length ratio of the template image OCR detection frame and the printed image OCR detection frame is calculated based on the complete matching detection frame. Four types of partial matching detection frames are set based on the prefix and suffix relationship of the corresponding text content in the template image OCR detection frame and the printed image OCR detection frame.

[0021] Expand the position of the missing string according to the average pixel length ratio based on the four types of partially matched frames, and search for intersecting detection frames in all detection frames. If there are no intersecting detection frames, add them to the list of partially matched detection frames;

[0022] If there are intersecting detection frames and the intersection ratio reaches the specified threshold, the detection frames are considered to belong to the same line of text. The text in the same line is merged into one detection frame, and the recognized text is merged at the same time. Character-level OCR is performed in the merged detection frame to locate the position of each character.

[0023] Calculate string similarity, locate the difference strings and their positions, set a string similarity threshold, and record the string matching boxes that meet the requirements according to the string similarity threshold as partial matching detection boxes.

[0024] In this solution, we post-process the text index position returned by PaddleOCR to obtain a separate detection box for each character, specifically:

[0025] Read the text column index and type information corresponding to the template image OCR results and the printed image OCR results, where Chinese characters occupy two columns and other characters occupy one column;

[0026] Calculate the width of each cell based on the width and number of columns of the template image OCR detection frame and the printed image OCR detection frame;

[0027] Calculate and save the detection box coordinates of English words based on the column index range. Calculate and save the detection box coordinates of Chinese characters one by one based on the character width. Add the content and detection box of each English word or Chinese character to the result list.

[0028] In this solution, in the visualization processing of the product label printed text detection results, the matching and non-matching text are marked with green and red boxes respectively, and the partially matched detection box will mark the different single characters.

[0029] The second aspect of the present invention provides a product label printed text detection system based on PaddleOCR, which includes a data input module, an OCR recognition module, a dynamic matching module and a visual output module;

[0030] The data input module is responsible for uploading and storing the template images and printed images corresponding to the product labels, and performing image preprocessing;

[0031] The OCR recognition module is responsible for performing OCR content recognition on the template image and the printed image using PaddleOCR, optimizing text splitting through dynamic adaptive merging of detection frames, and obtaining detection frames for each line of text;

[0032] The dynamic matching module is responsible for dynamically matching the detection frames in the template image and the printed image, and recording the detection frames that are fully matched, partially matched, and unmatched;

[0033] The visualization output module is responsible for marking the recognition record results of the detection frame with different preset colors to visualize the detection results of the product label printing text.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] (1) Improve proofreading efficiency: The dynamic adaptive merging algorithm of detection frames solves the problem of excessive splitting of a line of text by PaddleOCR, without the need for fine-tuning the model, significantly improving the versatility and stability of the method.

[0036] (2) Improve proofreading accuracy: Dynamic matching and character-level similarity calculation reduce manual intervention and significantly improve proofreading accuracy and production efficiency.

[0037] (3) Intuitive result display: Visual proofreading results help quickly locate different characters and positions, providing users with efficient and accurate proofreading support. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments or exemplary embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or exemplary descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained according to these drawings without paying any creative work.

[0039] Figure 1 The flowchart of the product label printing text detection method based on PaddleOCR is shown;

[0040] Figure 2 The figure shows the process flow of the product label printing text detection method based on PaddleOCR;

[0041] Figure 3 A flowchart of using an adaptive merging algorithm to solve the detection frame splitting problem is shown;

[0042] Figure 4 The block diagram of the product label printed text detection system based on PaddleOCR is shown. DETAILED DESCRIPTION

[0043] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0044] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0045] like Figure 1 As shown, the first embodiment of the present invention provides a method for detecting printed text on a product label based on PaddleOCR, including:

[0046] S102, uploading and storing the template image and printed image corresponding to the product label, and performing image preprocessing, performing OCR content recognition on the template image and the printed image, and obtaining detection frames for each line of text;

[0047] S104, dynamically matching the detection frames in the template image and the printed image, and recording detection frames that are fully matched, partially matched, and unmatched;

[0048] S106 , marking the detection frame with different preset colors according to the recognition record results, and visualizing the detection results of the product label printed text.

[0049] It should be noted that users upload and store template images and printed images corresponding to product labels through the interface and perform image preprocessing. The corresponding template image is obtained based on the target product label, uploaded, and the template name and internal persistent file name are stored in the database. The printed image corresponding to the target product label is obtained, uploaded, and stored in a temporary storage folder in the database. After the detection task is completed, the temporary storage folder is cleared to release storage resources. The stored template image and printed image are grayscaled, subjected to image noise reduction, image segmentation, tilt correction, and binarization to obtain the preprocessed printed image, thereby reducing useless information in the image and constructing the dataset to be identified.

[0050] like Figure 2As shown, the printed image and the persistent template image in the dataset to be recognized are used as input for OCR recognition. The PP-OCRv4 model is called in PaddleOCR to perform global OCR detection and recognition on the template image and the printed image. The DBNet pre-trained model is used to detect the text area in the image and output the detection frame of the text area. The text line image is cropped based on the detection frame, and the CRNN pre-trained model is used for text content recognition. The PP-LCNet model is used to detect the text direction. The text content recognition results are corrected for text direction, and the content of each line of text and the coordinate position of its detection frame are output. In PaddleOCR technology, the PP-OCRv4 model is used for text target detection and text content recognition, and the PP-LCNet model is used to detect text direction. This completes the three stages of text detection, direction recognition, and text recognition. Finally, the Paddle pre-trained weights are used for inference.

[0051] Use the re library to perform regular expression matching on specified symbols, convert the recognized characters in the template image and the printed image into full-width and half-width characters, and perform nested loops on the converted template image OCR results and the printed image OCR results lists to achieve pairwise comparison. Based on the paired comparison results, record the fully matched detection boxes, partially matched detection boxes, and unmatched detection boxes.

[0052] Figure 3 A flowchart of using an adaptive merging algorithm to solve the detection box splitting problem is shown.

[0053] According to an embodiment of the present invention, adaptive merging is used to split the detection box, solving the problem of a detection box splitting a line of text into two detection boxes, which causes matching failure. Character-level similarity analysis and dynamic expansion of the detection box are used to locate text differences. Specifically,

[0054] S302: Calculate the average pixel length ratio of the template image OCR detection frame and the printed image OCR detection frame based on the fully matched detection frame, and set four types of partially matched detection frames based on the prefix and suffix relationship of the corresponding text content in the template image OCR detection frame and the printed image OCR detection frame;

[0055] S304: Expand the position of the missing character string according to the average pixel length ratio based on the four types of partially matched frames, and search for intersecting detection frames among all detection frames. If there are no intersecting detection frames, add the missing character string to the list of partially matched detection frames.

[0056] S306: If there are intersecting detection frames and the intersection ratio reaches a specified threshold, the detection frames are considered to belong to the same line of text. The text in the same line is merged into one detection frame, and the recognized text is merged. Character-level OCR is performed in the merged detection frame to locate each character.

[0057] S308 , calculating string similarity, locating the difference strings and their positions, setting a string similarity threshold, and recording string matching boxes that meet the requirements according to the string similarity threshold as partial matching detection boxes.

[0058] It should be noted that a nested loop is performed on the two lists of template image OCR results and printed image OCR results. The recognized content of the detection frame in the template image is A, and the recognized content of the detection frame in the printed image is B. Four types of partially matching detection frames are screened out: A is the prefix string of B, B is the prefix string of A, A is the suffix string of B, and B is the suffix string of A. Based on the four types of partially matching detection frames, the partially matching detection frames are determined to meet the prefix or suffix rules. The direction in which the current detection frame needs to be expanded is determined. The missing portion is expanded using the average pixel length ratio, and the expanded detection frame is determined to intersect with other frames. When the intersection ratio reaches a preset threshold (for example, 50%), the detection frames are considered to belong to the same line of text. After merging, the coordinate range of the detection frames is recalculated, and the text content is concatenated into a complete line of text. The merged detection frame is compared with the matching detection frame to determine whether it meets the string similarity threshold. If the content matches exactly, it is added to the list of completely matching detection frames. If it does not match exactly but meets the similarity threshold, the different characters and their position index range are extracted and added to the list of partially matching detection frames.

[0059] Read the text index information returned by OCR, calculate the width and coordinate range of each character based on the detection box width, and process each character separately. Ultimately, the content and detection box coordinates of each character are obtained. String similarity is calculated, and then the specific difference characters are extracted. Ultimately, the content and index range of replaced characters, missing characters, and extra characters are obtained.

[0060] By post-processing the text index position returned by PaddleOCR, a separate detection frame for each character is obtained. Specifically, the following steps are taken: read the text column index and type information corresponding to the template image OCR result and the printed image OCR result, where Chinese characters occupy two columns and other characters occupy one column; calculate the width of each cell according to the width and number of columns of the template image OCR detection frame and the printed image OCR detection frame; calculate and save the detection frame coordinates of English words according to the column index range, calculate and save the detection frame coordinates of Chinese characters one by one according to the character width, generate detection frames for English words as a whole, and generate independent character detection frames for Chinese characters one by one; add the content and detection frame of each English word or Chinese character to the result list.

[0061] Preferably, while using PaddleOCR to perform OCR content recognition on the template image and the printed image, the template image and the printed image are processed by a dynamic threshold method, morphological processing is performed on the processed image to achieve preliminary screening of the text area, the area screened by morphological processing is filled, the area before and after the area is filled is compared to obtain the area ratio, the text area is identified according to the area ratio, and the corresponding detection frame is generated; the result list of the two content recognition methods is obtained, the detection frame coordinates therein are corrected, the difference in the vertical axis coordinates of the detection frames in the same row is eliminated, and the detection frames of the two content recognition methods are sorted according to the detection frame coordinates to generate a text block with consistent front and back order; after completing the sorting of the detection frames of the two content recognition methods, the detection frames of the two methods are traversed and matched, and the similarity is calculated using the length of the detection frame and the number of detection frame items, and the omission of the detection frame is judged according to the similarity. When the similarity does not meet the preset requirements, it proves that the detection frame obtained by PaddleOCR is missed, and the result list returned by PaddleOCR is corrected.

[0062] It should be noted that in the visualization of the printed text detection results on product labels, matching and non-matching text are marked with green and red boxes, respectively. In actual business operations, completely matching detection boxes are often not marked to facilitate quick location of differences, while partially matching detection boxes will use red marks to identify the individual characters with differences. The detection and annotation results of the printed text on product labels are output to the user through the interface, and the QR code scanning result information is added to the display according to actual business needs. At the same time, the original template image is provided for reference to facilitate quick comparison and proofreading.

[0063] Figure 4 The flowchart of the product label printed text detection system based on PaddleOCR is shown.

[0064] The second embodiment of the present invention provides a product label printed text detection system based on PaddleOCR, which includes a data input module 401, an OCR recognition module 402, a dynamic matching module 403 and a visual output module 404;

[0065] The data input module is responsible for uploading and storing the template images and printed images corresponding to the product labels, and performing image preprocessing;

[0066] The OCR recognition module is responsible for performing OCR content recognition on the template image and the printed image using PaddleOCR, optimizing text splitting through dynamic adaptive merging of detection frames, and obtaining detection frames for each line of text;

[0067] The dynamic matching module is responsible for dynamically matching the detection frames in the template image and the printed image, and recording the detection frames that are fully matched, partially matched, and unmatched;

[0068] The visualization output module is responsible for marking the recognition record results of the detection frame with different preset colors to visualize the detection results of the product label printing text.

[0069] The third embodiment of the present invention provides a computer-readable storage medium, which includes a product label printed text detection method program based on PaddleOCR. When the product label printed text detection method program based on PaddleOCR is executed by a processor, the steps of the product label printed text detection method based on PaddleOCR are implemented.

[0070] In the several embodiments provided in this application, it should be understood that the disclosed methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0071] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0072] Alternatively, if the integrated modules of the present invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (such as a personal computer, server, or network device) to execute all or part of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as removable storage devices, ROM, RAM, magnetic disks, or optical disks.

[0073] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed by the present invention, which should be covered by the scope of protection of the present invention.

Claims

1. A method for detecting printed text on product labels based on PaddleOCR, characterized in that: The following steps are involved: Upload and store the template image and printed image corresponding to the product label, pre-process the image, perform OCR content recognition on the template image and the printed image, and obtain the detection frame of each line of text; Dynamically match the detection frames in the template image and the printed image, and record the detection frames that are fully matched, partially matched, and unmatched; The recognition record results of the detection frame are marked with different preset colors to visualize the detection results of the printed text on the product label; Dynamically match the detection boxes in the template image and the printed image, specifically: Convert the recognized characters in the template image and the printed image into full-width and half-width characters, and perform nested loops on the converted template image OCR results and the printed image OCR results lists to achieve pairwise comparison; According to the paired comparison results, record the fully matched detection frame, partially matched detection frame, and unmatched detection frame; Adaptive merging is used to split the detection frame. Character-level similarity analysis and dynamic expansion of the detection frame are used to locate text differences. Specifically: The average pixel length ratio of the template image OCR detection frame and the printed image OCR detection frame is calculated based on the complete matching detection frame. Four types of partial matching detection frames are set based on the prefix and suffix relationship of the corresponding text content in the template image OCR detection frame and the printed image OCR detection frame. Expand the position of the missing string according to the average pixel length ratio based on the four types of partially matched frames, and search for intersecting detection frames in all detection frames. If there are no intersecting detection frames, add them to the list of partially matched detection frames; If there are intersecting detection frames and the intersection ratio reaches the specified threshold, the detection frames are considered to belong to the same line of text. The text in the same line is merged into one detection frame, and the recognized text is merged at the same time. Character-level OCR is performed in the merged detection frame to locate the position of each character. Calculate string similarity, locate the difference strings and their positions, set a string similarity threshold, and record the string matching boxes that meet the requirements according to the string similarity threshold as partial matching detection boxes; By post-processing the text index position returned by PaddleOCR, we can get a separate detection box for each character. Specifically: Read the text column index and type information corresponding to the template image OCR results and the printed image OCR results, where Chinese characters occupy two columns and other characters occupy one column; Calculate the width of each cell based on the width and number of columns of the template image OCR detection frame and the printed image OCR detection frame; Calculate and save the detection box coordinates of English words based on the column index range. Calculate and save the detection box coordinates of Chinese characters one by one based on the character width. Add the content and detection box of each English word or Chinese character to the result list.

2. The method for detecting printed text on product labels based on PaddleOCR according to claim 1, characterized in that: Upload and store the template image and print image corresponding to the product label, and pre-process the image, specifically: Obtain the corresponding template image according to the target product label, upload the template image and save it persistently in the database; Obtain a printed image corresponding to the target product label, upload the printed image and store it in a temporary storage folder in the database, and clean up the temporary storage folder after the detection task is completed; The stored template images and printed images are subjected to grayscale processing, image noise reduction, image segmentation, tilt correction and binarization processing respectively to obtain the preprocessed printed images and construct the data set to be identified.

3. The method for detecting printed text on product labels based on PaddleOCR according to claim 1, characterized in that: PaddleOCR is used to perform content recognition from the template image and the printed image, specifically: Call the PP-OCRv4 model in PaddleOCR to perform global OCR detection and recognition on the template image and the printed image. Use the DBNet pre-trained model to detect the text area in the image and output the detection box of the text area. The text line image is cropped according to the detection frame, the CRNN pre-trained model is used for text content recognition, the PP-LCNet model is used to detect the text direction, the text direction is corrected for the text content recognition result, and the content of each line of text and the coordinate position of its detection frame are output.

4. The method for detecting printed text on product labels based on PaddleOCR according to claim 1, characterized in that: In the visualization of the product label printed text detection results, matching and non-matching text are marked with green and red boxes respectively, while the partially matched detection box will mark the different single characters.

5. A product label printing text detection system based on PaddleOCR, characterized by: Used to implement the product label printed text detection method based on PaddleOCR as described in any one of claims 1-4, the system includes a data input module, an OCR recognition module, a dynamic matching module and a visual output module; The data input module is responsible for uploading and storing the template images and printed images corresponding to the product labels, and performing image preprocessing; The OCR recognition module is responsible for performing OCR content recognition on the template image and the printed image using PaddleOCR, optimizing text splitting through dynamic adaptive merging of detection frames, and obtaining detection frames for each line of text; The dynamic matching module is responsible for dynamically matching the detection frames in the template image and the printed image, and recording the detection frames that are fully matched, partially matched, and unmatched; The visualization output module is responsible for marking the recognition record results of the detection frame with different preset colors to visualize the detection results of the product label printing text.

Images