A method and apparatus for identifying paper version gift lists
By combining image correction and the ResNet network model, the problem of poor accuracy in vertical text recognition was solved, and the paper version of the gift book was digitized, making it easier to count and look up.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUIZHOU LEQI NETWORK TECH CO LTD
- Filing Date
- 2021-11-26
- Publication Date
- 2026-07-21
Smart Images
Figure CN116189186B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of text recognition technology, specifically relating to a method and apparatus for recognizing paper-based gift books. Background Technology
[0002] China is known as a land of etiquette, with diverse and rich folk customs. The exchange of gifts has always been an important part of daily life, passed down through generations. Whether it's a wedding or a funeral, gift-giving is indispensable, making gift books a crucial tool for recording monetary and monetary gifts. Traditionally, this was done manually, which was cumbersome and inconvenient for future reference, and also affected the flow of social obligations.
[0003] Gift books are mostly recorded vertically, while traditional character recognition is generally horizontal, which is less accurate for vertical text. Summary of the Invention
[0004] In view of this, the purpose of the present invention is to overcome the shortcomings of the prior art and provide a method and apparatus for recognizing paper gift books, so as to solve the problem of poor accuracy in recognizing vertical text in the prior art.
[0005] According to a first aspect of the embodiments of this application, a method for identifying a paper-based gift book is provided, the method comprising:
[0006] Collect original images of the printed gift book;
[0007] The original image is corrected to obtain the first intermediate image;
[0008] The first intermediate image is then subjected to text localization and text recognition in sequence to obtain the recognition results.
[0009] Furthermore, the step of correcting the original image to obtain the first intermediate image includes:
[0010] The original image is corrected using a preset image correction method, and noise is removed from the corrected original image to obtain a first intermediate image.
[0011] Furthermore, the step of correcting the original image using a preset image correction method includes:
[0012] The original image is sequentially converted to grayscale and then thresholded to obtain the second intermediate image;
[0013] Contour detection is performed on the second intermediate image to obtain the contour information of the second intermediate image;
[0014] Let the largest contour in the contour information be the bounding matrix, and obtain the tilt angle of the bounding matrix;
[0015] The second intermediate image is rotated and corrected according to the tilt angle of the enclosing matrix to obtain the first intermediate image.
[0016] Furthermore, the text localization of the first intermediate image includes:
[0017] The first intermediate image is binarized using the OTSU algorithm to obtain the third intermediate image;
[0018] The third intermediate image is horizontally projected to determine the upper and lower boundaries of each line of text in the third intermediate image, and each line of text in the third intermediate image is then segmented.
[0019] Vertical projection is performed on each line of text in the cut-out third intermediate image to determine the left and right boundaries of each character, and each character is cut to obtain the region position of each character in the third intermediate image.
[0020] Furthermore, the text recognition of the first intermediate image includes:
[0021] Based on the regional position of each character in the third intermediate image, the third intermediate image is identified using a preset ResNet network model to obtain a set of characters;
[0022] Based on the regional position of each character, each character in the character set is sorted to obtain the recognition result.
[0023] Furthermore, the method also includes:
[0024] Establish a pre-defined ResNet network model.
[0025] Furthermore, establishing the preset ResNet network model includes:
[0026] The CASIA-HWDB dataset is divided into a training set and a test set.
[0027] The ResNet network model is trained using the training set to obtain the trained ResNet network model;
[0028] The trained ResNet network model is validated using the test set. When the accuracy of the validation result is greater than or equal to the accuracy threshold, training is stopped, and the trained ResNet network model is the preset ResNet network model. When the accuracy of the validation result is less than the accuracy threshold, the parameters of the ResNet network model are adjusted, and the ResNet network model is retrained using the training set.
[0029] According to a second aspect of the embodiments of this application, an apparatus for identifying a paper-based gift book is provided, the apparatus comprising:
[0030] The data acquisition module is used to acquire original images of the printed gift book.
[0031] The correction module is used to correct the original image to obtain a first intermediate image;
[0032] The recognition module is used to sequentially locate and recognize text in the first intermediate image to obtain the recognition result.
[0033] Furthermore, the correction module is specifically used for:
[0034] The original image is corrected using a preset image correction method, and noise is removed from the corrected original image to obtain a first intermediate image.
[0035] Furthermore, the step of correcting the original image using a preset image correction method includes:
[0036] The original image is sequentially converted to grayscale and then thresholded to obtain the second intermediate image;
[0037] Contour detection is performed on the second intermediate image to obtain the contour information of the second intermediate image;
[0038] Let the largest contour in the contour information be the bounding matrix, and obtain the tilt angle of the bounding matrix;
[0039] The second intermediate image is rotated and corrected according to the tilt angle of the enclosing matrix to obtain the first intermediate image.
[0040] Furthermore, the recognition module includes: a text localization submodule and a text recognition submodule.
[0041] Furthermore, the text positioning submodule is specifically used for:
[0042] The first intermediate image is binarized using the OTSU algorithm to obtain the third intermediate image;
[0043] The third intermediate image is horizontally projected to determine the upper and lower boundaries of each line of text in the third intermediate image, and each line of text in the third intermediate image is then segmented.
[0044] Vertical projection is performed on each line of text in the cut-out third intermediate image to determine the left and right boundaries of each character, and each character is cut to obtain the region position of each character in the third intermediate image.
[0045] Furthermore, the text recognition submodule is specifically used for:
[0046] Based on the regional position of each character in the third intermediate image, the third intermediate image is identified using a preset ResNet network model to obtain a set of characters;
[0047] Based on the regional position of each character, each character in the character set is sorted to obtain the recognition result.
[0048] Furthermore, the device also includes:
[0049] The model building module is used to build a pre-defined ResNet network model.
[0050] Furthermore, the model building module is specifically used for:
[0051] The CASIA-HWDB dataset is divided into a training set and a test set.
[0052] The ResNet network model is trained using the training set to obtain the trained ResNet network model;
[0053] The trained ResNet network model is validated using the test set. When the accuracy of the validation result is greater than or equal to the accuracy threshold, training is stopped, and the trained ResNet network model is the preset ResNet network model. When the accuracy of the validation result is less than the accuracy threshold, the parameters of the ResNet network model are adjusted, and the ResNet network model is retrained using the training set.
[0054] The beneficial effects achieved by adopting the above technical solution in this invention include: by collecting the original image of the paper gift book, correcting the original image to obtain a first intermediate image, and performing text positioning and text recognition on the first intermediate image in sequence to obtain the recognition result, the paper gift book is digitized, which facilitates the calculation of amounts and future reference. Attached Figure Description
[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0056] Figure 1 This is a flowchart illustrating a method for identifying a paper gift book according to an exemplary embodiment;
[0057] Figure 2 This is a structural block diagram of a device for recognizing paper gift books, according to an exemplary embodiment. Detailed Implementation
[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be described in detail below. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this invention.
[0059] Figure 1 This is a flowchart illustrating a method for identifying a paper gift book according to an exemplary embodiment, such as... Figure 1 As shown, this method can be used in a terminal, but is not limited to, and includes the following steps:
[0060] Step 101: Collect the original image of the printed gift book;
[0061] Step 102: Correct the original image to obtain the first intermediate image;
[0062] Step 103: Perform text localization and text recognition on the first intermediate image in sequence to obtain the recognition results.
[0063] In some embodiments, the original image of the paper gift book can be captured, but is not limited to, through a camera device or a smart terminal such as a mobile phone.
[0064] This invention provides a method for recognizing paper gift books. By acquiring the original image of the paper gift book, correcting the original image to obtain a first intermediate image, and then performing text localization and text recognition on the first intermediate image to obtain the recognition result, the paper gift book is digitized, which facilitates the calculation of amounts and future reference.
[0065] Furthermore, step 102 includes:
[0066] The original image is corrected using a preset image correction method, and noise is removed from the corrected original image to obtain the first intermediate image.
[0067] In some embodiments, the preset image correction method may be implemented, but is not limited to, through a contour extraction-based correction algorithm.
[0068] It should be noted that the method of "removing noise from the corrected original image" involved in the embodiments of the present invention is well known to those skilled in the art, therefore, its specific implementation will not be described in detail.
[0069] In some alternative embodiments, step 102 may be implemented by, but is not limited to, the following process:
[0070] Step 1021: Perform grayscale conversion and threshold binarization on the original image sequentially to obtain the second intermediate image;
[0071] Step 1022: Perform contour detection on the second intermediate image to obtain the contour information of the second intermediate image;
[0072] Step 1023: Let the largest contour in the contour information be the bounding matrix, and obtain the tilt angle of the bounding matrix;
[0073] Step 1024: Rotate and correct the second intermediate image according to the tilt angle of the enclosing matrix to obtain the first intermediate image.
[0074] In some embodiments, the enclosing matrix can be represented, but is not limited to, two-dimensional or three-dimensional data. It is understood that the largest contour in the contour information is generally the outermost contour of the second intermediate image. The contour information generally includes at least the coordinates of the top, bottom, left, and right corners of the image.
[0075] Understandably, since the images to be recognized may be messy, tilted, or noisy, image preprocessing is necessary to correct the text position and remove noise before further text localization and recognition can be performed.
[0076] Furthermore, step 103 involves locating text in the first intermediate image, including:
[0077] The first intermediate image is binarized using the OTSU algorithm to obtain the third intermediate image;
[0078] The third intermediate image is horizontally projected to determine the upper and lower boundaries of each line of text in the third intermediate image, and each line of text in the third intermediate image is then segmented.
[0079] Vertical projection is performed on each line of text in the cut-out third intermediate image to determine the left and right boundaries of each character, and each character is then cut to obtain the region position of each character in the third intermediate image.
[0080] Understandably, horizontal projection involves counting the elements in each row of an image (that is, counting horizontally), and then drawing a statistical result graph based on the statistical results to determine the start and end points of each row.
[0081] It should be noted that the "horizontal projection" and "vertical projection" methods involved in the embodiments of the present invention are well known to those skilled in the art, therefore, their specific implementation methods will not be described in detail.
[0082] Furthermore, step 103 involves text recognition of the first intermediate image, including:
[0083] Based on the region location of each character in the third intermediate image, the pre-set ResNet network model is used to identify the third intermediate image and obtain the character set;
[0084] Based on the location of each character in the region, each character in the character set is sorted to obtain the recognition result.
[0085] Understandably, sorting each character in the text set according to its regional position will result in the same recognition order as the paper gift book, making it easier to distinguish each person's name and the amount of the gift. This allows users to more easily track gift amounts and conveniently check records of social interactions in the future.
[0086] It should be noted that by locating the text before text recognition and then sorting it according to the location (i.e., coordinates) of the text, it is possible to correctly locate and recognize text, whether it is horizontal or vertical.
[0087] Furthermore, the method also includes:
[0088] Establish a pre-defined ResNet network model.
[0089] Furthermore, a pre-defined ResNet network model is established, including:
[0090] The CASIA-HWDB dataset is divided into a training set and a test set.
[0091] The ResNet network model is trained using the training set to obtain the trained ResNet network model;
[0092] The trained ResNet network model is validated using a test set. When the accuracy of the validation result is greater than or equal to the accuracy threshold, training is stopped, and the trained ResNet network model becomes the preset ResNet network model. When the accuracy of the validation result is less than the accuracy threshold, the parameters of the ResNet network model are adjusted, and the ResNet network model is retrained using the training set.
[0093] It is understood that the ResNet network model is a residual network model, and those skilled in the art can set the number of layers in the ResNet network model based on experimental data, etc. In some embodiments, the ResNet50 network model may be used, but is not limited to.
[0094] For example, the CASIA-HWDB dataset contains 3755 Chinese characters. Let's assume the CASIA-HWDB dataset includes these 3755 characters written by 100 people, meaning 100 different ways of writing these 3755 characters. Let the 3755 characters written by 80 people be the training set, and the 3755 characters written by the remaining 20 people be the test set. Use the training set to train a ResNet network model, obtaining the trained ResNet network model. Use the test set to validate the trained ResNet network model. When the validation accuracy is greater than or equal to 99%, training stops, and the trained ResNet network model becomes the preset ResNet network model. When the validation accuracy is less than 99%, adjust the parameters of the ResNet network model or adjust the ResNet network model itself, and then retrain the ResNet network model using the training set again.
[0095] It should be noted that the embodiments of the present invention do not limit the "accuracy threshold" and "adjustment of ResNet network model parameters," which can be set by those skilled in the art based on experimental data or expert experience. In some embodiments, the accuracy threshold may be, but is not limited to, 99%.
[0096] This invention provides a method for recognizing paper gift books. By acquiring the original image of the paper gift book, correcting the original image to obtain a first intermediate image, and then performing text localization and text recognition on the first intermediate image to obtain the recognition result, the paper gift book is digitized, which facilitates the calculation of amounts and future reference.
[0097] To complement the above-described method for identifying paper gift books, this invention provides an apparatus for identifying paper gift books, as described above. Figure 2 The device includes:
[0098] The data acquisition module is used to acquire original images of the printed gift book.
[0099] The correction module is used to correct the original image to obtain the first intermediate image;
[0100] The recognition module is used to perform text localization and text recognition on the first intermediate image in sequence to obtain the recognition result.
[0101] Furthermore, the correction module is specifically used for:
[0102] The original image is corrected using a preset image correction method, and noise is removed from the corrected original image to obtain the first intermediate image.
[0103] Furthermore, the original image is corrected using a preset image correction method, including:
[0104] The original image is sequentially converted to grayscale and then thresholded for binarization to obtain the second intermediate image.
[0105] Contour detection is performed on the second intermediate image to obtain its contour information;
[0106] Let the largest contour in the contour information be the bounding matrix, and obtain the tilt angle of the bounding matrix;
[0107] The second intermediate image is rotated and corrected based on the tilt angle of the enclosing matrix to obtain the first intermediate image.
[0108] Furthermore, the recognition module includes: a text localization submodule and a text recognition submodule.
[0109] Furthermore, the text positioning submodule is specifically used for:
[0110] The first intermediate image is binarized using the OTSU algorithm to obtain the third intermediate image;
[0111] The third intermediate image is horizontally projected to determine the upper and lower boundaries of each line of text in the third intermediate image, and each line of text in the third intermediate image is then segmented.
[0112] Vertical projection is performed on each line of text in the cut-out third intermediate image to determine the left and right boundaries of each character, and each character is then cut to obtain the region position of each character in the third intermediate image.
[0113] Furthermore, the text recognition submodule is specifically used for:
[0114] Based on the region location of each character in the third intermediate image, the pre-set ResNet network model is used to identify the third intermediate image and obtain the character set;
[0115] Based on the location of each character in the region, each character in the character set is sorted to obtain the recognition result.
[0116] Furthermore, the device also includes:
[0117] The model building module is used to build a pre-defined ResNet network model.
[0118] Furthermore, the model building module is specifically used for:
[0119] The CASIA-HWDB dataset is divided into a training set and a test set.
[0120] The ResNet network model is trained using the training set to obtain the trained ResNet network model;
[0121] The trained ResNet network model is validated using a test set. When the accuracy of the validation result is greater than or equal to the accuracy threshold, training is stopped, and the trained ResNet network model becomes the preset ResNet network model. When the accuracy of the validation result is less than the accuracy threshold, the parameters of the ResNet network model are adjusted, and the ResNet network model is retrained using the training set.
[0122] This invention provides a device for recognizing paper gift books. The device acquires the original image of the paper gift book through an acquisition module, corrects the original image to obtain a first intermediate image, and then performs text positioning and text recognition on the first intermediate image to obtain the recognition result. This device digitizes the paper gift book, making it easier to calculate the amount and check it later.
[0123] It is understood that the device embodiments provided above correspond to the method embodiments described above, and the specific details can be referred to each other, which will not be repeated here.
[0124] This invention also provides a readable storage medium storing an executable program thereon, which, when executed by a processor, implements the steps in the method for recognizing a paper gift book provided in the above embodiments.
[0125] It is understood that the readable storage medium provided in this embodiment of the invention can be implemented, but is not limited to, by creating an APP. The APP can be installed on a smart terminal such as a mobile phone, thereby enabling the recognition of paper gift books through the smart terminal such as a mobile phone. This greatly facilitates users in calculating gift money and makes it easier to query records of social interactions in the future.
[0126] In some optional embodiments, the original image of the paper gift book can be captured by a smart terminal, and then the original image is corrected by the server to obtain a first intermediate image. Then, the server performs text localization and text recognition on the first intermediate image in sequence to obtain the recognition result.
[0127] It is understood that the above-provided embodiments correspond to each other, and the specific details can be referred to each other, which will not be repeated here.
[0128] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.
[0129] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0130] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction methods implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0131] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0132] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A method for identifying a paper gift book, characterized in that, The method includes: Collect original images of the printed gift book; The original image is corrected to obtain the first intermediate image; The step of correcting the original image to obtain the first intermediate image includes: The original image is corrected using a preset image correction method, and noise is removed from the corrected original image to obtain a first intermediate image; The step of correcting the original image using a preset image correction method includes: The original image is sequentially converted to grayscale and then thresholded to obtain the second intermediate image; Contour detection is performed on the second intermediate image to obtain the contour information of the second intermediate image; Let the largest contour in the contour information be the bounding matrix, and obtain the tilt angle of the bounding matrix; The second intermediate image is rotated and corrected according to the tilt angle of the enclosing matrix to obtain the first intermediate image; The first intermediate image is sequentially processed for text localization and text recognition to obtain the recognition results; The text localization of the first intermediate image includes: The first intermediate image is binarized using the OTSU algorithm to obtain the third intermediate image; The third intermediate image is horizontally projected to determine the upper and lower boundaries of each line of text in the third intermediate image, and each line of text in the third intermediate image is then segmented. Vertical projection is performed on each line of text in the cut-out third intermediate image to determine the left and right boundaries of each character, and each character is cut to obtain the region position of each character in the third intermediate image; The text recognition process for the first intermediate image includes: Based on the regional position of each character in the third intermediate image, the third intermediate image is identified using a preset ResNet network model to obtain a set of characters; Based on the regional position of each character, each character in the character set is sorted to obtain the recognition result.
2. The method according to claim 1, characterized in that, The method further includes: Establish a pre-defined ResNet network model.
3. The method according to claim 2, characterized in that, The establishment of the preset ResNet network model includes: The CASIA-HWDB dataset is divided into a training set and a test set. The ResNet network model is trained using the training set to obtain the trained ResNet network model; The trained ResNet network model is validated using the test set. When the accuracy of the validation result is greater than or equal to the accuracy threshold, training is stopped, and the trained ResNet network model is the preset ResNet network model. When the accuracy of the validation result is less than the accuracy threshold, the parameters of the ResNet network model are adjusted, and the ResNet network model is retrained using the training set.
4. A device for identifying paper gift books, characterized in that, The device includes: The data acquisition module is used to acquire original images of the printed gift book. The correction module is used to correct the original image to obtain a first intermediate image; The step of correcting the original image to obtain the first intermediate image includes: The original image is corrected using a preset image correction method, and noise is removed from the corrected original image to obtain a first intermediate image; The step of correcting the original image using a preset image correction method includes: The original image is sequentially converted to grayscale and then thresholded to obtain the second intermediate image; Contour detection is performed on the second intermediate image to obtain the contour information of the second intermediate image; Let the largest contour in the contour information be the bounding matrix, and obtain the tilt angle of the bounding matrix; The second intermediate image is rotated and corrected according to the tilt angle of the enclosing matrix to obtain the first intermediate image; The recognition module is used to perform text localization and text recognition on the first intermediate image sequentially to obtain the recognition result; The text localization of the first intermediate image includes: The first intermediate image is binarized using the OTSU algorithm to obtain the third intermediate image; The third intermediate image is horizontally projected to determine the upper and lower boundaries of each line of text in the third intermediate image, and each line of text in the third intermediate image is then segmented. Vertical projection is performed on each line of text in the cut-out third intermediate image to determine the left and right boundaries of each character, and each character is cut to obtain the region position of each character in the third intermediate image; The text recognition process for the first intermediate image includes: Based on the regional position of each character in the third intermediate image, the third intermediate image is identified using a preset ResNet network model to obtain a set of characters; Based on the regional position of each character, each character in the character set is sorted to obtain the recognition result.