Commodity label printing character detection method and system based on PaddleOCR

Abstract

The invention discloses a PaddleOCR-based commodity label printed character detection method and system, and relates to the technical field of quality inspection automation, and the method comprises the steps: uploading and storing a template picture and a printed picture corresponding to a commodity label, carrying out the OCR content recognition of the template picture and the printed picture, and obtaining a detection frame of each line of characters; dynamically matching the detection frames in the template picture and the printing picture, and recording completely matched, partially matched and unmatched detection frames; according to the identification record result of the detection frame, different preset colors are used for identification, and commodity label printing character detection result visualization is carried out. According to the method, the template and the printing content are recognized through the OCR, the similarity of the detection frame is dynamically calculated, character-level difference positioning is combined, a visual result is provided, and the method is suitable for quality verification in the label printing industry.

Description

Technical Field

[0001] The present invention relates to the field of quality inspection automation technology, and more specifically, to a method and system for detecting printed text on a product label based on PaddleOCR. Background Art

[0002] With the rapid development of the product label printing industry, the types and quantity of labels are increasing. In the process of label production and printing, due to factors such as production efficiency and process complexity, problems such as label content deviation, misprinting and missing printing are prone to occur. These errors not only affect the display effect of the product, but may also cause consumer confusion, damage the brand image, and even lead to legal disputes.

[0003] Traditional label proofreading methods mainly rely on manual work, and differences are found by manually comparing templates with printed labels one by one. However, this method has the following problems: Large workload: The label content is complex and the number is large. Manual proofreading is time-consuming and labor-intensive, and it is difficult to meet the needs of large-scale production. Unstable accuracy: Manual proofreading is easily affected by factors such as fatigue and subjective judgment differences, and the error rate is high. Low efficiency: Under high-precision requirements, the efficiency of manual proofreading is difficult to synchronize with the production rhythm. Existing OCR technology has problems such as line text splitting, character matching failure, and inaccurate difference positioning in label proofreading scenarios. PaddleOCR is an open source OCR (optical character recognition) tool library launched by Baidu PaddlePaddle. It supports multi-language text detection, recognition and layout analysis, and has the characteristics of high precision, lightweight, and easy deployment. In view of the large workload and instability of traditional manual proofreading methods, an efficient and accurate proofreading method based on PaddleOCR technology is urgently needed. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a product label printed text detection method and system based on PaddleOCR. Through the strategies of adaptive merging detection frames, dynamic matching and similarity calculation, the problems of splitting a line of text and locating differences are optimized, which greatly improves the proofreading efficiency and accuracy and reduces labor costs.

[0005] The present invention provides a method for detecting printed text on a product label based on PaddleOCR, comprising the following steps: Upload and store the template image and printed image corresponding to the product label, perform image preprocessing, perform OCR content recognition on the template image and the printed image, and obtain the detection frame of each line of text; Dynamically match the detection frames in the template image and the printed image, and record the detection frames that are fully matched, partially matched, and unmatched; According to the recognition record results of the detection frame, different preset colors are used to mark them respectively to visualize the detection results of the product label printing text.

[0006] In this solution, the template image and printed image corresponding to the product label are uploaded and stored, and the image is preprocessed, specifically: Obtain the corresponding template image according to the target product label, upload the template image and save it persistently in the database; Obtain a printed image corresponding to the target product label, upload the printed image and store it in a temporary storage folder in the database, and clean up the temporary storage folder after the inspection task is completed; The stored template images and printed images are subjected to grayscale processing, image noise reduction, image segmentation, tilt correction and binarization processing respectively to obtain the preprocessed printed images and construct the data set to be identified.

[0007] In this solution, PaddleOCR is used to perform content recognition from template images and printed images, specifically: Call the PP-OCRv4 model in PaddleOCR to perform global OCR detection and recognition on the template image and the printed image, use the DBNet pre-trained model to detect the text area in the image, and output the detection box of the text area; The text line image is cropped according to the detection frame, the CRNN pre-trained model is used for text content recognition, the PP-LCNet model is used to detect the text direction, the text content recognition result is corrected for the text direction, and the content of each line of text and the coordinate position of its detection frame are output.

[0008] In this solution, the detection frames in the template image and the printed image are dynamically matched, specifically: Convert the recognized characters in the template image and the printed image into full-width and half-width characters, and perform nested loops on the converted template image OCR results and the printed image OCR results lists to achieve pairwise comparison; According to the paired comparison results, the fully matched detection frame, partially matched detection frame and unmatched detection frame are recorded.

[0009] In this solution, adaptive merging is used to split the detection frame, and the text differences are located through character-level similarity analysis and dynamic expansion of the detection frame. Specifically: The average pixel length ratio between the template image OCR detection frame and the printed image OCR detection frame is calculated based on the complete matching detection frame, and four types of partial matching detection frames are set based on the prefix and suffix relationship of the corresponding text content in the template image OCR detection frame and the printed image OCR detection frame; According to the four types of partial matching frames, the position of the missing character string is expanded according to the average pixel length ratio, and whether there is an intersecting detection frame in all detection frames is searched, and if there is no intersecting detection frame, it is added to the partial matching detection frame list; If there are intersecting detection frames and the intersection ratio reaches the specified threshold, the detection frames are considered to belong to the same line of text. The same line of text is merged into one detection frame, and its recognized text is merged at the same time. Character-level OCR is performed in the merged detection frame to locate the position of each character. Calculate string similarity, locate the difference string and its position, set the string similarity threshold, and record the string matching box that meets the requirements as a partial matching detection box according to the string similarity threshold.

[0010] In this solution, the text index position returned by PaddleOCR is post-processed to obtain a separate detection box for each character, specifically: Read the text column index and type information corresponding to the template image OCR result and the printed image OCR result, where Chinese characters occupy two columns and other characters occupy one column; Calculate the width of each cell based on the width and number of columns of the template image OCR detection frame and the printed image OCR detection frame; Calculate and save the detection box coordinates of English words according to the column index range, calculate and save the detection box coordinates of Chinese characters one by one according to the character width, and add the content and detection box of each English word or Chinese character to the result list.

[0011] In this solution, in the visualization processing of the product label printed text detection results, the matching and non-matching text are marked with green and red boxes respectively, and the partially matched detection box marks the different single characters.

[0012] The second aspect of the present invention provides a product label printed text detection system based on PaddleOCR, the system comprising a data input module, an OCR recognition module, a dynamic matching module and a visual output module; The data input module is responsible for uploading and storing the template images and printed images corresponding to the product labels, and performing image preprocessing; The OCR recognition module is responsible for performing OCR content recognition on the template image and the printed image using PaddleOCR, optimizing text splitting through dynamic adaptive merging of detection frames, and obtaining detection frames for each line of text; The dynamic matching module is responsible for dynamically matching the detection frames in the template image and the printed image, and recording the detection frames that are fully matched, partially matched, and unmatched; The visualization output module is responsible for marking the recognition record results of the detection frame with different preset colors to visualize the detection results of the product label printing text.

[0013] Compared with the prior art, the present invention has the following beneficial effects: (1) Improve proofreading efficiency: The detection box dynamic adaptive merging algorithm solves the problem of PaddleOCR splitting too much when detecting a line of text. There is no need to fine-tune the model, which significantly improves the versatility and stability of the method.

[0014] (2) Improve proofreading accuracy: Dynamic matching and character-level similarity calculation reduce manual intervention and significantly improve proofreading accuracy and production efficiency.

[0015] (3) Intuitive result display: Visual proofreading results help quickly locate different characters and positions, providing users with efficient and accurate proofreading support. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments or exemplary embodiments of the present invention, the drawings required for use in the embodiments or exemplary descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained according to the drawings without paying creative work.

[0017] Figure 1 The flowchart of the product label printing text detection method based on PaddleOCR is shown; Figure 2 The flowchart of the product label printing text detection method based on PaddleOCR is shown; Figure 3 A flow chart showing the use of an adaptive merging algorithm to solve the problem of splitting detection frames; Figure 4 A block diagram of a product label printed text detection system based on PaddleOCR is shown. DETAILED DESCRIPTION

[0018] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0019] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0020] like Figure 1 As shown, the first embodiment of the present invention provides a method for detecting printed text on a product label based on PaddleOCR, comprising: S102, uploading and storing the template image and the printed image corresponding to the product label, and performing image preprocessing, performing OCR content recognition on the template image and the printed image, and obtaining the detection frame of each line of text; S104, dynamically matching the detection frames in the template image and the printed image, and recording detection frames that are completely matched, partially matched, and unmatched; S106, marking the detection frame with different preset colors according to the recognition record results, and visualizing the detection results of the product label printed text.

[0021] It should be noted that the user uploads and stores the template image and print image corresponding to the product label through the interface, and pre-processes the image. According to the target product label, the corresponding template image is obtained, the template image is uploaded and the template name and the file name of the internal persistent storage are stored in the database; the print image corresponding to the target product label is obtained, the print image is uploaded and stored in a temporary storage folder in the database, and the temporary storage folder is cleaned after the detection task is completed to release storage resources; the stored template image and print image are grayed, image denoised, image segmented, tilted and binarized to obtain the pre-processed print image, so as to reduce useless information in the image and construct the data set to be identified.

[0022] like Figure 2 As shown in the figure, the printed image in the dataset to be identified and the template image that is saved permanently are used as the input of OCR recognition. The PP-OCRv4 model is called in PaddleOCR to perform global OCR detection and recognition on the template image and the printed image. The DBNet pre-trained model is used to detect the text area in the image and output the detection frame of the text area. The text line image is cropped according to the detection frame, and the CRNN pre-trained model is used for text content recognition. The PP-LCNet model is used to detect the text direction, and the text content recognition result is corrected for the text direction. The content of each line of text and the coordinate position of its detection frame are output. In PaddleOCR technology, the PP-OCRv4 model is used for text target detection and text content recognition, and the PP-LCNet model is used to detect the text direction. The three stages of text detection, direction recognition, and text recognition are completed. Finally, the Paddle pre-trained weights are used for inference.

[0023] Use the re library to perform regular matching of specified symbols, convert the recognized characters in the template image and the printed image into full-width and half-width characters, and perform nested loops on the two lists of the converted template image OCR results and the printed image OCR results to achieve pairwise comparison; record the complete match detection box, partial match detection box, and non-match detection box based on the pairwise comparison results.

[0024] Figure 3 A flowchart for solving the detection frame splitting problem using an adaptive merging algorithm is shown.

[0025] According to an embodiment of the present invention, adaptive merging is used to split the detection frame to solve the problem that the detection frame splits a line of text into two detection frames, resulting in matching failure. The text differences are located through character-level similarity analysis and dynamic expansion of the detection frame, specifically: S302, calculating the average pixel length ratio of the template image OCR detection frame and the printed image OCR detection frame according to the complete matching detection frame, and setting four types of partial matching detection frames according to the prefix and suffix relationship of the corresponding text content in the template image OCR detection frame and the printed image OCR detection frame; S304, expanding the position of the missing character string according to the average pixel length ratio based on the four types of partial matching frames, searching whether there is an intersecting detection frame among all the detection frames, and adding the intersecting detection frame to the partial matching detection frame list if there is no intersecting detection frame; S306, if there are intersecting detection frames and the intersection ratio reaches a specified threshold, the detection frames are considered to belong to the same line of text, and the same line of text is merged into one detection frame, and the recognized text is merged at the same time, and character-level OCR is performed in the merged detection frame to locate each character position; S308, calculating the string similarity, locating the difference string and its position, setting the string similarity threshold, and recording the string matching box that meets the requirement as a partial matching detection box according to the string similarity threshold.

[0026] It should be noted that the two lists of template image OCR results and printed image OCR results are nested and looped. The recognition content of the detection box in the template image is A, and the recognition content of the detection box in the printed image is B. Four types of partially matched detection boxes are screened out, namely A is the prefix string of B, B is the prefix string of A, A is the suffix string of B, and B is the suffix string of A. According to the set four types of partially matched detection boxes, the partially matched detection box is judged to meet the prefix or suffix rules, the direction in which the current detection box needs to be extended is determined, the missing part is extended using the average pixel length ratio, and it is judged whether the extended detection box has an intersection relationship with other boxes. When the intersection ratio reaches the preset threshold (for example: 50%), it is considered to belong to the same line of text. After merging, the coordinate range of the detection box is recalculated, and the text content is spliced ​​into a complete line of text. It is judged whether the content comparison between the merged detection box and the matching detection box meets the string similarity threshold. If the content is completely matched, it is added to the list of completely matched detection boxes. If it is not a complete match but meets the similarity threshold, the difference characters and their position index range are extracted and added to the list of partially matched detection boxes.

[0027] Read the text index information returned by OCR, calculate the width and coordinate range of each character according to the detection box width, and process them separately according to the character type. Finally, obtain the content and detection box coordinates of each character. Perform string similarity calculation, then extract the specific difference characters, and finally obtain the content and index range of replaced characters, missing characters, and redundant characters.

[0028] By post-processing the text index position returned by PaddleOCR, a separate detection frame for each character is obtained. Specifically, the text column index and type information corresponding to the template image OCR result and the printed image OCR result are read, where Chinese characters occupy two columns and other characters occupy one column; the width of each cell is calculated according to the width and number of columns of the template image OCR detection frame and the printed image OCR detection frame; the detection frame coordinates of the English words are calculated and saved according to the column index range, and the detection frame coordinates of the Chinese characters are calculated and saved one by one according to the character width; a detection frame is generated for the English words as a whole, and an independent character detection frame is generated for the Chinese characters one by one; the content and detection frame of each English word or Chinese character are added to the result list.

[0029] Preferably, while using PaddleOCR to perform OCR content recognition on the template image and the printed image, the template image and the printed image are processed by a dynamic threshold method, morphological processing is performed on the processed image to achieve preliminary screening of the text area, the area screened by the morphological processing is filled, the area before and after the area is filled is compared to obtain the area ratio, the text area is identified according to the area ratio, and a corresponding detection frame is generated; a result list of the two content recognition methods is obtained, the detection frame coordinates therein are corrected, the difference in the vertical axis coordinates of the detection frames in the same row is eliminated, and the detection frames of the two content recognition methods are sorted according to the detection frame coordinates to generate a text block with a consistent front-to-back order; after completing the sorting of the detection frames of the two content recognition methods, the detection frames of the two methods are traversed and matched, the length of the detection frame and the number of detection frame items are used to calculate the similarity, and the omission of the detection frame is judged according to the similarity. When the similarity does not meet the preset requirements, it proves that the detection frame obtained by PaddleOCR has omissions, and the result list returned by PaddleOCR is corrected.

[0030] It should be noted that in the visualization of the results of the product label printing text detection, the matching and non-matching text are marked with green and red boxes respectively. In actual business, the detection boxes that are completely matched are often not marked, which is convenient for quickly locating the difference position, and the detection boxes that are partially matched will use red marks to mark the single characters with differences. The detection and annotation results of the product label printing text are output to the user through the interface, and the QR code scanning result information display is added according to the actual business needs. At the same time, the original template image is given for reference, which is convenient for quick comparison and proofreading.

[0031] Figure 4 The flowchart of the product label printed text detection system based on PaddleOCR is shown.

[0032] The second embodiment of the present invention provides a product label printed text detection system based on PaddleOCR, the system includes a data input module 401, an OCR recognition module 402, a dynamic matching module 403 and a visual output module 404; The data input module is responsible for uploading and storing the template images and printed images corresponding to the product labels, and performing image preprocessing; The OCR recognition module is responsible for performing OCR content recognition on the template image and the printed image using PaddleOCR, optimizing text splitting through dynamic adaptive merging of detection frames, and obtaining detection frames for each line of text; The dynamic matching module is responsible for dynamically matching the detection frames in the template image and the printed image, and recording the detection frames that are fully matched, partially matched, and unmatched; The visualization output module is responsible for marking the recognition record results of the detection frame with different preset colors to visualize the detection results of the product label printing text.

[0033] The third embodiment of the present invention provides a computer-readable storage medium, which includes a product label printed text detection method program based on PaddleOCR. When the product label printed text detection method program based on PaddleOCR is executed by a processor, the steps of the product label printed text detection method based on PaddleOCR are implemented.

[0034] In the several embodiments provided in the present application, it should be understood that the disclosed method can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0035] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0036] Alternatively, if the above-mentioned integrated module of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0037] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for detecting printed text on product labels based on PaddleOCR, characterized in that: The following steps are involved: Upload and store the template image and printed image corresponding to the product label, perform image preprocessing, perform OCR content recognition on the template image and the printed image, and obtain the detection frame of each line of text; Dynamically match the detection frames in the template image and the printed image, and record the detection frames that are fully matched, partially matched, and unmatched; According to the recognition record results of the detection frame, different preset colors are used to mark them respectively to visualize the detection results of the product label printing text.

2. The method for detecting printed text on a product label based on PaddleOCR according to claim 1, characterized in that: Upload and store the template images and print images corresponding to the product labels, and pre-process the images, specifically: Obtain the corresponding template image according to the target product label, upload the template image and save it persistently in the database; Obtain a printed image corresponding to the target product label, upload the printed image and store it in a temporary storage folder in the database, and clean up the temporary storage folder after the inspection task is completed; The stored template images and printed images are subjected to grayscale processing, image noise reduction, image segmentation, tilt correction and binarization processing respectively to obtain the preprocessed printed images and construct the data set to be identified.

3. The method for detecting printed text on a product label based on PaddleOCR according to claim 1, characterized in that: PaddleOCR is used to perform content recognition from template images and printed images, specifically: Call the PP-OCRv4 model in PaddleOCR to perform global OCR detection and recognition on the template image and the printed image, use the DBNet pre-trained model to detect the text area in the image, and output the detection box of the text area; The text line image is cropped according to the detection frame, the CRNN pre-trained model is used for text content recognition, the PP-LCNet model is used to detect the text direction, the text content recognition result is corrected for the text direction, and the content of each line of text and the coordinate position of its detection frame are output.

4. The method for detecting printed text on a product label based on PaddleOCR according to claim 1, characterized in that: Dynamically match the detection frames in the template image and the printed image, specifically: Convert the recognized characters in the template image and the printed image into full-width and half-width characters, and perform nested loops on the converted template image OCR results and the printed image OCR results lists to achieve pairwise comparison; According to the paired comparison results, the fully matched detection frame, partially matched detection frame and unmatched detection frame are recorded.

5. The method for detecting printed text on a product label based on PaddleOCR according to claim 4, characterized in that: Adaptive merging is used to split the detection frame, and the text differences are located through character-level similarity analysis and dynamic expansion of the detection frame. Specifically: The average pixel length ratio between the template image OCR detection frame and the printed image OCR detection frame is calculated based on the complete matching detection frame, and four types of partial matching detection frames are set based on the prefix and suffix relationship of the corresponding text content in the template image OCR detection frame and the printed image OCR detection frame; According to the four types of partial matching frames, the position of the missing character string is expanded according to the average pixel length ratio, and whether there is an intersecting detection frame in all detection frames is searched, and if there is no intersecting detection frame, it is added to the partial matching detection frame list; If there are intersecting detection frames and the intersection ratio reaches the specified threshold, the detection frames are considered to belong to the same line of text. The same line of text is merged into one detection frame, and its recognized text is merged at the same time. Character-level OCR is performed in the merged detection frame to locate the position of each character. Calculate string similarity, locate the difference string and its position, set the string similarity threshold, and record the string matching box that meets the requirements as a partial matching detection box according to the string similarity threshold.

6. The method for detecting printed text on a product label based on PaddleOCR according to claim 5, characterized in that: By post-processing the text index position returned by PaddleOCR, we can get a separate detection box for each character, specifically: Read the text column index and type information corresponding to the template image OCR result and the printed image OCR result, where Chinese characters occupy two columns and other characters occupy one column; Calculate the width of each cell based on the width and number of columns of the template image OCR detection frame and the printed image OCR detection frame; Calculate and save the detection box coordinates of English words according to the column index range, calculate and save the detection box coordinates of Chinese characters one by one according to the character width, and add the content and detection box of each English word or Chinese character to the result list.

7. The method for detecting printed text on a product label based on PaddleOCR according to claim 1, characterized in that: In the visualization of the product label printed text detection results, the matching and non-matching text are marked with green and red boxes respectively, and the partially matched detection box marks the different single characters.

8. A product label printing text detection system based on PaddleOCR, characterized in that: Used to implement the commodity label printed text detection method based on PaddleOCR as described in any one of claims 1-7, the system includes a data input module, an OCR recognition module, a dynamic matching module and a visual output module; The data input module is responsible for uploading and storing the template images and printed images corresponding to the product labels, and performing image preprocessing; The OCR recognition module is responsible for performing OCR content recognition on the template image and the printed image using PaddleOCR, optimizing text splitting through dynamic adaptive merging of detection frames, and obtaining detection frames for each line of text; The dynamic matching module is responsible for dynamically matching the detection frames in the template image and the printed image, and recording the detection frames that are fully matched, partially matched, and unmatched; The visualization output module is responsible for marking the recognition record results of the detection frame with different preset colors to visualize the detection results of the product label printing text.

Images