Text recognition method, apparatus, device, medium, and product
By setting target recognition templates and mapping location information, the problem of needing to retrain the OCR model after updating the certificate type is solved, achieving efficient text recognition and saving time and resources.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA CONSTRUCTION BANK
- Filing Date
- 2023-01-04
- Publication Date
- 2026-07-07
AI Technical Summary
Existing technologies require retraining the OCR model after the certificate type is updated, which wastes time and resources and results in low efficiency.
By adopting a generalized recognition model, a target recognition template is set, which includes the location information of the anchor area and the area to be recognized. Text information is compared and mapped to determine the mapping position of the area to be recognized, thereby realizing text recognition of different types of certificates.
It enables text recognition on different types of certificates and licenses, saving time and resources and improving recognition efficiency.
Smart Images
Figure CN116092066B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of information extraction technology, and in particular relates to a text recognition method, device, equipment, medium and product. Background Technology
[0002] With the continuous development of deep learning technology, optical character recognition (OCR) devices, as a text recognition technology, are widely used in daily life for text recognition and extraction of images based on OCR devices.
[0003] When extracting textual information from certificates with standard formats, different OCR models need to be developed for different certificates. Whenever the certificate type is updated, the recognition models for different types of certificates need to be retrained, which wastes a lot of time and resources. Summary of the Invention
[0004] This application provides a text recognition method, apparatus, device, medium, and product that can perform text recognition on different types of certificates based on a generalized recognition model, saving a significant amount of time and resources.
[0005] In a first aspect, embodiments of this application provide a text recognition method, the method comprising:
[0006] Set up a target recognition template for the target type of document. The target recognition template includes standard text information and anchor point area location information corresponding to multiple anchor point areas, as well as target area location information corresponding to the area to be recognized.
[0007] Obtain the target document image corresponding to the target document, perform text recognition on the target document image, and obtain the initial recognition result. The initial recognition result includes text information and text region location information corresponding to multiple text regions respectively.
[0008] When the type of the target document is the target type, the text information in the initial recognition result is compared one by one with the standard text information in the target recognition template to determine multiple pairs of target text regions and target anchor point regions that match the text information.
[0009] Based on the mapping relationship between the text region location information corresponding to the target text region and the anchor region location information corresponding to the target anchor region, the mapping region location information corresponding to the mapping region of the region to be identified in the target document image is determined according to the target region location information.
[0010] Based on the location information of the mapped region and the location information of the text regions corresponding to the multiple text regions, determine the first text region that intersects with the mapped region from the multiple text regions;
[0011] The first text information corresponding to the first text region is obtained from the initial recognition result, and the recognition result corresponding to the target document image is output based on the first text information.
[0012] Secondly, embodiments of this application provide a text recognition device, which includes:
[0013] The setting module is used to set target recognition templates for target type documents. The target recognition template includes standard text information and anchor point area location information corresponding to multiple anchor point areas, as well as target area location information corresponding to the area to be recognized.
[0014] The acquisition module is used to acquire the target document image corresponding to the target document, perform text recognition on the target document image, and obtain the initial recognition result. The initial recognition result includes text information and text region location information corresponding to multiple text regions respectively.
[0015] The first determination module is used to compare the text information in the initial recognition result with the standard text information in the target recognition template one by one when the type of the target document is the target type, and to determine multiple pairs of target text regions and target anchor point regions that match the text information.
[0016] The second determining module is used to determine the mapping region location information corresponding to the mapping region of the region to be identified in the target document image based on the mapping relationship between the text region location information corresponding to the target text region and the anchor region location information corresponding to the target anchor region.
[0017] The third determining module is used to determine the first text region that intersects with the mapping region from multiple text regions based on the mapping region location information and the text region location information corresponding to the multiple text regions respectively;
[0018] The output module is used to obtain the first text information corresponding to the first text region from the initial recognition result, and output the recognition result corresponding to the target document image based on the first text information.
[0019] Thirdly, embodiments of this application provide an electronic device, which includes: a processor and a memory storing computer program instructions;
[0020] When the processor executes the computer program instructions, it implements the steps of the text recognition method as described in any embodiment of the first aspect.
[0021] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the steps of the text recognition method as described in any embodiment of the first aspect.
[0022] Fifthly, embodiments of this application provide a computer program product in which instructions, when executed by a processor of an electronic device, cause the electronic device to perform the steps of the text recognition method as described in any embodiment of the first aspect.
[0023] The text recognition method, apparatus, device, medium, and product in this application embodiment set a recognition template for the target type of document. The template contains anchor points corresponding to standard text information and anchor point location information, as well as target region location information corresponding to the region to be recognized. Then, by acquiring the text information and text region location information from the initial text recognition result of the target document image, after determining the type of the target document, the text information is compared with the standard text information to determine multiple pairs of matching target text regions and target anchor point regions. Based on the mapping relationship between the text region location information of the target text region and the anchor region location information corresponding to the target anchor region, the mapping location information corresponding to the mapped region of the region to be recognized in the target document image can be determined according to the target region location information. Based on the mapped region location information and the text region location information corresponding to the text regions, the first text region intersecting with the mapped region can be determined. The first text information corresponding to the first text region is output as the recognition result corresponding to the target document image. When performing text recognition on different documents, based on the anchor point region and region to be recognized information set in the template, text recognition of different types of documents based on a universal recognition template is achieved, saving a significant amount of time and resources. Attached Figure Description
[0024] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0025] Figure 1 This is a flowchart illustrating a text recognition method provided in an embodiment of this application;
[0026] Figure 2 This is a schematic diagram of a text recognition method provided in an embodiment of this application;
[0027] Figure 3 This is a schematic diagram of the structure of a text recognition device provided in an embodiment of this application;
[0028] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0029] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.
[0030] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.
[0031] The acquisition, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.
[0032] When extracting textual information from certificates with standard formats, different OCR models need to be developed for different certificates. Whenever the certificate type is updated, data annotation and recognition model development need to be carried out again, and the recognition model needs to be retrained, which wastes a lot of time and resources.
[0033] To address the problems in the prior art, embodiments of this application provide a text recognition method, apparatus, device, medium, and product.
[0034] The text recognition method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.
[0035] Figure 1 This is a flowchart illustrating a text recognition method provided in an embodiment of this application. Figure 1 As shown, the text recognition method may specifically include the following steps:
[0036] S110. Set a target recognition template for the target type of document. The target recognition template includes standard text information and anchor point area location information corresponding to multiple anchor point areas, as well as target area location information corresponding to the area to be recognized.
[0037] S120. Obtain the target document image corresponding to the target document, perform text recognition on the target document image, and obtain the initial recognition result. The initial recognition result includes text information and text region location information corresponding to multiple text regions respectively.
[0038] S130. When the type of the target document is the target type, the text information in the initial recognition result is compared with the standard text information in the target recognition template one by one to determine multiple pairs of target text regions and target anchor point regions that match the text information.
[0039] S140. Based on the mapping relationship between the text region location information corresponding to the target text region and the anchor region location information corresponding to the target anchor region, determine the mapping region location information corresponding to the mapping region of the region to be identified in the target document image according to the target region location information.
[0040] S150. Based on the location information of the mapped region and the location information of the text regions corresponding to the multiple text regions, determine the first text region that intersects with the mapped region from the multiple text regions;
[0041] S160. Obtain the first text information corresponding to the first text region from the initial recognition result, and output the recognition result corresponding to the target document image based on the first text information.
[0042] Therefore, by setting a recognition template for the target type of document, the anchor point regions in the template correspond to standard text information and anchor point region location information, as well as the target region location information corresponding to the region to be recognized. Then, by acquiring the text information and text region location information from the initial text recognition result of the target document image, after determining the type of the target document, the text information is compared with the standard text information to identify multiple pairs of matching target text regions and target anchor point regions. Based on the mapping relationship between the text region location information of the target text region and the anchor region location information corresponding to the target anchor point region, the mapping location information of the region to be recognized in the target document image can be determined according to the target region location information. Based on the mapping region location information and the text region location information corresponding to the text regions, the first text region intersecting with the mapping region can be determined. The first text information corresponding to the first text region is output as the recognition result corresponding to the target document image. When performing text recognition on different types of documents, based on the anchor point regions and the region to be recognized information set in the template, text recognition of different types of documents based on a universal recognition template is achieved, saving a significant amount of time and resources.
[0043] The specific implementation methods for each of the above steps are described below.
[0044] In some implementations, during S110, the target type of document involved in the embodiments of this application can be a standard format document such as an ID card or business license. The target recognition template can be set with multiple anchor points corresponding to the target type of document. These anchor points can include, for example, anchor point names, multiple regular expressions configured for the anchor points, and anchor point positions.
[0045] In addition, the standard text information corresponding to multiple anchor point areas can include fixed text and regular expressions. The fixed text can be text that remains constant on the document, such as the fixed text on an ID card containing "Name," "Gender," "Ethnicity," "Birth," "Address," and "Citizen ID Number." The area to be recognized can be a variable text content area corresponding to the standard text information; there can be multiple areas to be recognized. The area location information can be its coordinates on the template, such as the top-left and bottom-right coordinates. These four coordinates can be obtained from the standard rectangle in the template.
[0046] As an example, such as Figure 2 As shown, the solid box represents the anchor point area, and the dashed box represents the area to be identified.
[0047] In some embodiments, the anchor point region may include a fixed text region, a variable text region, and a combination of fixed and variable text regions.
[0048] As an example, the fixed text area in the anchor area can be, for example, the fixed text "gender" in an ID card; the variable text area can be, for example, "male or female" corresponding to the fixed text "gender"; and the combined area of fixed and variable text can be, for example, "gender male or female".
[0049] In some implementations, in S120, the target document image corresponding to the target document can be any document image with a fixed standard format, such as an image of the back of an ID card.
[0050] In some embodiments, S120 may specifically include: inputting the target document image into a general optical character recognition (OCR) module, using the general OCR module to perform text recognition on the target document image, and outputting the initial recognition result.
[0051] As an example, when recognizing text information on the back of an ID card, a general OCR recognition device is used to perform text recognition on the document image to obtain initial recognition results, including text information corresponding to multiple text regions and text region location information in the document image. For example, the text "female" and the four sets of coordinate information corresponding to the text. Here, the coordinate information is based on the coordinate system of the document image.
[0052] In some implementations, in S130, multiple pairs of target text regions and target anchor regions that match the text information can be, for example, multiple pairs of text regions in the initial recognition results and anchor regions in the template obtained based on the matched text information, and the matched text regions and anchor regions are respectively used as target text regions and target anchor regions.
[0053] As an example, when recognizing text information on the back of an ID card, the initial recognition result from a general OCR device is compared one-to-one with the standard text information in the target recognition template. This yields the text information "birth year x month x day" and the coordinates of the text area in the initial recognition result regarding birth and date, as well as the coordinates and regular expressions of the anchor point area in the target template related to birth and date. Similarly, the text information and coordinates of the matching address and ID number, along with the coordinates and regular expressions of the anchor point area, can be obtained.
[0054] In some implementations, in S140, the mapping relationship involved in the embodiments of this application can be obtained by transforming the coordinates corresponding to the text region position coordinate information and the anchor point region position coordinate information through perspective transformation to obtain coordinates in a unified coordinate system. For example, the anchor point region position information is transformed by perspective transformation, so that the mapping region position coordinate information corresponding to the mapping region of the region to be identified in the target document image can be determined according to the target region position information.
[0055] Based on this, the aforementioned S140 may specifically include:
[0056] Based on the mapping relationship between the text region location information corresponding to the target text region and the anchor region location information corresponding to the target anchor region, the target recognition template is transformed according to the target document image to obtain the transformed target recognition template.
[0057] Based on the transformed target recognition template, the transformed region location information corresponding to the target region location information is determined, which is the mapping region location information corresponding to the mapping region of the region to be recognized in the target document image.
[0058] As an example, a transformation matrix can be determined based on the text region location information and the anchor point region location information. Based on the transformation matrix, the coordinate information corresponding to the target recognition template can be transformed to the coordinate system corresponding to the target document image. After obtaining the transformed target recognition template, the transformed region location information corresponding to the target region location information in the target recognition template can be determined. The transformed region location information is used as the mapping region location information corresponding to the mapping region of the region to be recognized in the target document image.
[0059] Therefore, by using the mapping relationship between the text region location information and the anchor point region location information, the mapping region location relationship of the region to be recognized in the target document image can be determined. This eliminates the need to crop the image twice to unify the image size before performing OCR text recognition again, effectively improving the efficiency of text recognition.
[0060] In some implementations, in S150, the first text region involved in the embodiments of this application can be the region where the mapped region location information and the location information of multiple text regions intersect. Text information can be obtained based on the intersecting first text region.
[0061] In some implementations, in S160, the target recognition template may further include a text layout corresponding to the area to be recognized. This text layout may be, for example, a single-line text format or a multi-line text format. A single-line text format may be, for example, the text information "name," "gender," "ethnicity," and "birth" found on an ID card. A multi-line text format may be, for example, the text information "address" found on an ID card.
[0062] In some embodiments, when there are multiple first text regions, in order to more accurately obtain multi-line text recognition results, the above-mentioned S160 may specifically include:
[0063] When the text layout is in single-line text format, determine the first target proportion of the intersection of the first text region and the mapping region to the mapping region.
[0064] The text region with the largest proportion of the first target is selected from multiple first text regions and used as the second text region.
[0065] Obtain the second text information corresponding to the second text region from the initial recognition results;
[0066] The recognition result corresponding to the target document image is output based on the second text information.
[0067] As an example, when selecting the first text information corresponding to the first text region, if the text layout is a single-line text, the first target ratio of the intersection of the first text region and the mapping region to the mapping region is determined. The text region with the largest first target ratio is obtained from multiple first text regions based on the first target ratio. The text region that matches the mapping region is then obtained as the second text region. The text information in the second text region is used as the text recognition result corresponding to the target document image.
[0068] Based on this, in some embodiments, in order to obtain more accurate text recognition results in multi-line format, the above-mentioned S160 may further include:
[0069] When the text layout is in the form of multi-line text, determine the second target proportion of the intersection of the first text region and the mapping region to the first text region;
[0070] One or more text regions whose second target ratio is greater than a preset threshold are obtained from multiple first text regions and used as second text regions;
[0071] Obtain the second text information corresponding to the second text region from the initial recognition results;
[0072] When there is only one second text region, the recognition result corresponding to the target document image is output based on the second text information.
[0073] When there are multiple second text regions, the second text information corresponding to the multiple text regions is concatenated to obtain third text information, and the recognition result corresponding to the target document image is output based on the third text information.
[0074] In some embodiments, the preset threshold involved in this application may be the minimum value of the intersection of the first text region and the mapped region within the first text region, wherein the preset threshold may be set to, for example, 0.5. The second text information may be one or more text information corresponding to the second text region identified when the text layout is multi-line text.
[0075] As an example, when selecting the first text information corresponding to the first text region, if the text layout is multi-line text, a second target ratio is determined for the intersection of the first text region and the mapped region. One or more text regions with a second target ratio greater than 0.5 are obtained as the second text region. Text regions matching the mapped region are then obtained as the second text region, and the text information in the second text region is used as the text recognition result corresponding to the target document image. If only one second text region is obtained, the text information corresponding to the second text region is output as the recognition result of the target document image. If multiple second text regions are obtained, the multiple text information pieces are concatenated from top to bottom and left to right to obtain complete text information as the third text information, which is then used as the text recognition result corresponding to the target document image.
[0076] In some embodiments, the target recognition template described above also includes prefix information and / or suffix information corresponding to the area to be recognized. This prefix and suffix information can be used to remove any "to" information from the text information corresponding to the area to be recognized, and the prefix and / or suffix information need to be set accordingly based on different certificate types.
[0077] Based on this, in some embodiments, the above-mentioned output of the recognition result corresponding to the target document image based on the first text information may specifically include:
[0078] The redundant information contained in the first text information is determined based on the prefix information and / or suffix information;
[0079] Remove redundant information from the first text information and output the recognition result corresponding to the target document image.
[0080] As an example, when recognizing an image as an ID card image, prefix and / or suffix information can be set accordingly based on the information in the ID card image. For example, the prefix information for the name in the area to be recognized is "Name," and the suffix information is "Ethnicity." The prefix information for the ethnicity in the area to be recognized is "Gender: Male / Female Ethnicity." However, the "Birth," "Address," and "Citizen ID Number," which are on separate lines, do not require prefix and / or suffix information. When the obtained first text information contains redundant information, such as the first text corresponding to the ethnicity in the area to be recognized being "Han Woman," the redundant information "Female" can be deleted according to the set prefix and / or suffix information, and then the first text can be output as the recognition result of the target ID card image.
[0081] In some embodiments, when the prefix information and / or suffix information include multiple fixed characters and there is no regular expression, the determination of redundant information contained in the first text information based on the prefix information and / or suffix information may specifically include:
[0082] Multiple character sets are obtained by arranging and combining multiple fixed characters;
[0083] The characters in the first text information are compared with the characters contained in multiple character sets to determine the redundant information contained in the first text information.
[0084] As an example, in cases where the prefix and / or suffix information includes multiple fixed characters and there are no regular expressions, such as the fixed text "name" contained in standard text information, the combination of multiple fixed characters is obtained, and multiple character sets are obtained by arranging and combining multiple fixed characters. The characters in the first text information are compared with the multiple fixed characters in the multiple character sets to determine the redundant information contained in the first text information.
[0085] In some embodiments, when the prefix information and / or suffix information include regular expressions, the determination of redundant information contained in the first text information based on the prefix information and / or suffix information may specifically include:
[0086] The multiple optional characters corresponding to the regular expression are compared with the characters in the first text information to determine the redundant information contained in the first text information.
[0087] As an example, when the prefix and / or suffix information includes regular expressions, the set of multiple optional characters corresponding to the regular expression can be used to compare the characters in the first text information, and the characters in the first text information that match the most fixed characters can be selected to determine the redundant information contained in the first text information.
[0088] In some embodiments, the above-mentioned target recognition template further includes a plurality of target optional characters corresponding to the area to be recognized; the initial recognition result further includes candidate text information corresponding to the plurality of text areas respectively.
[0089] Based on this, in some embodiments, in order to output a correct recognition result when the target optional characters in the initial recognition result are inaccurate, the above-mentioned outputting the recognition result corresponding to the target document image according to the first text information includes:
[0090] In the case where the first text information does not include characters identical to the target optional characters, obtain the first candidate text information corresponding to the first text area from the initial recognition result;
[0091] In the case where the first candidate text information includes characters identical to the target optional characters, output the recognition result corresponding to the target document image according to the first candidate text information;
[0092] In the case where the first candidate text information does not include characters identical to the target optional characters, determine the target character most similar to the characters included in the first text information from the plurality of target optional characters, and replace the characters included in the first text information with the target character, and output the recognition result corresponding to the target document image.
[0093] In some embodiments, the candidate text information involved in the embodiments of the present application may be multiple text information corresponding to the first text area in the initial recognition result, and the first text candidate information closest to the target optional word is determined based on the multiple candidate text information. The target optional characters may be characters with fixed optional options in the area to be recognized in any certificate, for example, "male / female" in the area to be recognized of the gender of the ID card.
[0094] As an example, when the recognition image is an ID card image, in the case where the first text information does not include characters identical to the target optional characters, for example, the text "female" corresponding to the area to be recognized of the gender is recognized as "wen", at this time, the first candidate text information corresponding to the first text area in the initial recognition result can be obtained. In the case where the first candidate text information contains the target optional word "female", output the first candidate text information as the recognition result corresponding to the target document image; in the case where the first candidate text does not include characters identical to the target candidate word, determine the target character "female" most similar to the first text information "wen" from the multiple target optional words "male / female", and thus replace the characters included in the first text information with the target character "female", and output the recognition result corresponding to the target document image.
[0095] In some embodiments, the target recognition template above also includes target character rules corresponding to the region to be recognized; the initial recognition result also includes candidate text information corresponding to multiple text regions respectively;
[0096] Based on this, in some embodiments, in order to obtain recognition results that conform to the target character rules, the above-mentioned output of the recognition result corresponding to the target document image based on the first text information includes:
[0097] Detect whether the characters included in the first text information conform to the target character rules;
[0098] If the characters included in the first text information do not conform to the target character rules, the first candidate text information corresponding to the first text region is obtained from the initial recognition result;
[0099] Detect whether the characters included in the first candidate text information conform to the target character rules;
[0100] If the characters included in the first candidate text information conform to the target character rules, the recognition result corresponding to the target document image is output based on the first candidate text information.
[0101] In some embodiments, the target character rules involved in the present application can be rules for any certificate information, such as the number of digits in an ID card number and the meaning represented by the corresponding digits.
[0102] As an example, when recognizing an image as an ID card image, the system detects whether the characters contained in the first text information conform to the rules for target characters. For example, in text information corresponding to an ID number, the system checks whether the ID number in the text information conforms to the rules for ID number recognition. If it does not conform, the system obtains the first candidate text information corresponding to the first text region from the initial recognition result. The system then checks whether the characters included in the first candidate text information conform to the rules for ID number recognition. If they do conform, the ID number in the first candidate text information is output as the recognition result for the target ID card image.
[0103] It should be noted that the application scenarios described in the above embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.
[0104] Based on the same inventive concept, this application also provides a text recognition device. Specifically, in conjunction with... Figure 3 Please provide a detailed explanation.
[0105] Figure 3This is a schematic diagram of the structure of a text recognition device provided in one embodiment of this application.
[0106] like Figure 3 As shown, the text recognition device 300 may include:
[0107] Setting module 301 is used to set a target recognition template for the target type of document. The target recognition template includes standard text information and anchor point area location information corresponding to multiple anchor point areas, as well as target area location information corresponding to the area to be recognized.
[0108] The acquisition module 302 is used to acquire the target document image corresponding to the target document, perform text recognition on the target document image, and obtain an initial recognition result. The initial recognition result includes text information and text region location information corresponding to multiple text regions respectively.
[0109] The first determining module 303 is used to compare the text information in the initial recognition result with the standard text information in the target recognition template one by one when the type of the target document is the target type, and to determine multiple pairs of target text regions and target anchor point regions that match the text information.
[0110] The second determining module 304 is used to determine the mapping region position information corresponding to the mapping region of the region to be identified in the target document image based on the mapping relationship between the text region position information corresponding to the target text region and the anchor region position information corresponding to the target anchor region.
[0111] The third determining module 305 is used to determine the first text region that intersects with the mapping region from multiple text regions based on the mapping region location information and the text region location information corresponding to the multiple text regions respectively.
[0112] The output module 306 is used to obtain the first text information corresponding to the first text region from the initial recognition result, and output the recognition result corresponding to the target document image based on the first text information.
[0113] The text recognition device 300 described above will be explained in detail below:
[0114] In some embodiments, the aforementioned multiple anchor point regions include fixed text regions, variable text regions, and regions combining fixed and variable text.
[0115] In some embodiments, the target recognition template may also include a text layout format corresponding to the region to be recognized.
[0116] Based on this, when there are multiple first text regions, in order to more accurately obtain multi-line text recognition results, the output module 306 may specifically include:
[0117] The first determining submodule is used to determine the first target proportion of the intersection of the first text region and the mapping region to the mapping region when the text layout is in the form of a single-line text.
[0118] The first acquisition submodule is used to acquire the text region with the largest proportion of the first target from multiple first text regions, and use it as the second text region;
[0119] The second acquisition submodule is used to acquire the second text information corresponding to the second text region from the initial recognition result;
[0120] The first output submodule is used to output the recognition result corresponding to the target document image based on the second text information.
[0121] In some embodiments, in order to obtain more accurate text recognition results in multi-line format, the output module 306 may specifically include:
[0122] The second determining submodule is used to determine the second target proportion of the intersection of the first text region and the mapping region to the first text region when the text layout is in the form of multi-line text.
[0123] The third acquisition submodule is used to acquire one or more text regions from multiple first text regions where the proportion of the second target is greater than a preset threshold, and use them as second text regions.
[0124] The fourth acquisition submodule is used to obtain the second text information corresponding to the second text region from the initial recognition results;
[0125] The second output submodule is used to output the recognition result corresponding to the target document image based on the second text information when the number of second text regions is one.
[0126] The third output submodule is used to concatenate the second text information corresponding to the multiple text regions when there are multiple second text regions to obtain third text information, and output the recognition result corresponding to the target document image based on the third text information.
[0127] In some embodiments, the target recognition template may further include prefix information and / or suffix information corresponding to the region to be recognized.
[0128] Based on this, the output module 306 mentioned above may specifically include:
[0129] The third determination submodule is used to determine the redundant information contained in the first text information based on the prefix information and / or suffix information.
[0130] The fourth output submodule is used to remove redundant information from the first text information and output the recognition result corresponding to the target document image.
[0131] In some embodiments, when the aforementioned prefix information and / or suffix information includes multiple fixed characters and no regular expression, the aforementioned third determining submodule may specifically include:
[0132] The permutation subunit is used to arrange and combine multiple fixed characters to obtain multiple character sets;
[0133] The first determining subunit is used to compare the characters in the first text information with the characters contained in multiple character sets to determine the redundant information contained in the first text information.
[0134] In some embodiments, where the prefix information and / or suffix information include regular expressions, the third determining submodule may specifically include:
[0135] The second determining subunit is used to compare the multiple optional characters corresponding to the regular expression with the characters in the first text information to determine the redundant information contained in the first text information.
[0136] In some embodiments, in order to output a correct recognition result when the initial recognition result is inaccurate for the target optional character, the output module 306 may specifically include:
[0137] The fifth acquisition submodule is used to acquire the first candidate text information corresponding to the first text region from the initial recognition result when the first text information does not contain any characters that are the same as the target optional characters.
[0138] The fifth output submodule is used to output the recognition result corresponding to the target document image based on the first candidate text information when the first candidate text information includes characters that are the same as the target optional characters.
[0139] The sixth output submodule is used to determine the target character most similar to the character included in the first text information from multiple target optional characters when the first candidate text information does not contain the same character as the target optional character, and replace the character included in the first text information with the target character, and output the recognition result corresponding to the target document image.
[0140] In some embodiments, in order to obtain recognition results that conform to the target character rules, the output module 306 may specifically include:
[0141] The first detection submodule is used to detect whether the characters included in the first text information conform to the target character rules;
[0142] The sixth acquisition submodule is used to acquire the first candidate text information corresponding to the first text region from the initial recognition result when the characters included in the first text information do not conform to the target character rules.
[0143] The second detection submodule is used to detect whether the characters included in the first candidate text information conform to the target character rules;
[0144] The seventh output submodule is used to output the recognition result corresponding to the target document image based on the first candidate text information, provided that the characters included in the first candidate text information conform to the target character rules.
[0145] In some embodiments, the second determining module 304 described above may specifically include:
[0146] The perspective transformation submodule is used to perform perspective transformation on the target recognition template according to the target document image based on the mapping relationship between the text region position information corresponding to the target text region and the anchor region position information corresponding to the target anchor point region, so as to obtain the transformed target recognition template.
[0147] The fourth determination submodule is used to determine the transformed region location information corresponding to the target region location information based on the transformed target recognition template. This is the mapped region location information corresponding to the mapped region of the region to be recognized in the target document image.
[0148] Therefore, by setting a recognition template for the target type of document, the anchor point regions in the template correspond to standard text information and anchor point region location information, as well as the target region location information corresponding to the region to be recognized. Then, by acquiring the text information and text region location information from the initial text recognition result of the target document image, after determining the type of the target document, the text information is compared with the standard text information to identify multiple pairs of matching target text regions and target anchor point regions. Based on the mapping relationship between the text region location information of the target text region and the anchor region location information corresponding to the target anchor point region, the mapping location information of the region to be recognized in the target document image can be determined according to the target region location information. Based on the mapping region location information and the text region location information corresponding to the text regions, the first text region intersecting with the mapping region can be determined. The first text information corresponding to the first text region is output as the recognition result corresponding to the target document image. When performing text recognition on different types of documents, based on the anchor point regions and the region to be recognized information set in the template, text recognition of different types of documents based on a universal recognition template is achieved, saving a significant amount of time and resources.
[0149] Figure 4 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application.
[0150] The electronic device 400 may include a processor 401 and a memory 402 storing computer program instructions.
[0151] Specifically, the processor 401 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.
[0152] Memory 402 may include mass storage for data or instructions. For example, and not limitingly, memory 402 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 402 may include removable or non-removable (or fixed) media. Where appropriate, memory 402 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 402 is non-volatile solid-state memory.
[0153] In certain embodiments, the memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Thus, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this application.
[0154] The processor 401 implements any of the text recognition methods described in the above embodiments by reading and executing computer program instructions stored in the memory 402.
[0155] In some examples, electronic device 400 may also include communication interface 403 and bus 410. For example, Figure 4 As shown, the processor 401, memory 402, and communication interface 403 are connected through bus 410 and complete communication with each other.
[0156] The communication interface 403 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.
[0157] Bus 410 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not as a limitation, bus 410 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 410 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.
[0158] For example, the electronic device 400 can be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc.
[0159] The electronic device 400 can execute the text recognition method in the embodiments of this application, thereby achieving a combination of Figure 1 and Figure 3 The text recognition method described.
[0160] Furthermore, in conjunction with the text recognition methods in the above embodiments, this application embodiment can provide a computer-readable storage medium for implementation. This computer-readable storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the text recognition methods in the above embodiments. Examples of computer-readable storage media include non-transitory computer-readable storage media, such as portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, etc.
[0161] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.
[0162] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.
[0163] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.
[0164] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.
[0165] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.
Claims
1. A text recognition method, characterized in that, include: A target recognition template is set for the target type of document. The target recognition template includes standard text information and anchor point area location information corresponding to multiple anchor point areas, as well as target area location information corresponding to the area to be recognized. Obtain the target document image corresponding to the target document, perform text recognition on the target document image, and obtain an initial recognition result. The initial recognition result includes text information and text region location information corresponding to multiple text regions respectively. When the type of the target document is the target type, the text information in the initial recognition result is compared one by one with the standard text information in the target recognition template to determine multiple pairs of target text regions and target anchor point regions that match the text information; Based on the mapping relationship between the text region location information corresponding to the target text region and the anchor region location information corresponding to the target anchor region, the mapping region location information corresponding to the mapping region of the region to be identified in the target document image is determined according to the target region location information. Based on the location information of the mapped region and the location information of the text regions corresponding to the plurality of text regions, a first text region intersecting with the mapped region is determined from the plurality of text regions; Obtain first text information corresponding to the first text region from the initial recognition result, and output the recognition result corresponding to the target document image based on the first text information; The target recognition template also includes a text layout format corresponding to the region to be recognized; When there are multiple first text regions, the step of obtaining first text information corresponding to the first text region from the initial recognition result and outputting the recognition result corresponding to the target document image based on the first text information includes: When the text layout is in the form of a single-line text, a first target proportion of the portion where the first text region intersects with the mapped region is determined to be the first target proportion of the mapped region. The text region with the largest proportion of the first target is selected from multiple first text regions and used as the second text region; Obtain the second text information corresponding to the second text region from the initial recognition result; Output the recognition result corresponding to the target document image based on the second text information; The step of obtaining first text information corresponding to the first text region from the initial recognition result and outputting the recognition result corresponding to the target document image based on the first text information includes: When the text layout is in the form of multi-line text, a second target proportion is determined for the portion of the first text region where the first text region intersects with the mapping region. One or more text regions whose second target ratio is greater than a preset threshold are obtained from multiple first text regions and used as second text regions; Obtain the second text information corresponding to the second text region from the initial recognition result; When the number of the second text regions is one, the recognition result corresponding to the target document image is output based on the second text information; When there are multiple second text regions, the second text information corresponding to the multiple text regions is concatenated to obtain third text information, and the recognition result corresponding to the target document image is output based on the third text information.
2. The method according to claim 1, characterized in that, The multiple anchor point regions include fixed text regions, variable text regions, and regions combining fixed and variable text.
3. The method according to claim 1, characterized in that, The target recognition template also includes prefix information and / or suffix information corresponding to the region to be recognized; The step of outputting the recognition result corresponding to the target document image based on the first text information includes: The redundant information contained in the first text information is determined based on the prefix information and / or suffix information; The redundant information in the first text information is deleted, and the recognition result corresponding to the target document image is output.
4. The method according to claim 3, characterized in that, When the prefix information and / or the suffix information include multiple fixed characters and there is no regular expression, determining the redundant information contained in the first text information based on the prefix information and / or the suffix information includes: By arranging and combining the multiple fixed characters, multiple character sets are obtained; The characters in the first text information are compared with the characters contained in the plurality of character sets to determine the redundant information contained in the first text information.
5. The method according to claim 3, characterized in that, When the prefix information and / or the suffix information include regular expressions, determining the redundant information contained in the first text information based on the prefix information and / or the suffix information includes: The multiple optional characters corresponding to the regular expression are compared with the characters in the first text information to determine the redundant information contained in the first text information.
6. The method according to claim 1, characterized in that, The target recognition template also includes multiple optional target characters corresponding to the region to be recognized; the initial recognition result also includes candidate text information corresponding to multiple text regions respectively; The step of outputting the recognition result corresponding to the target document image based on the first text information includes: If the first text information does not contain any characters that are the same as the target optional characters, the first candidate text information corresponding to the first text region is obtained from the initial recognition result; If the first candidate text information includes a character that is the same as the target selectable character, the recognition result corresponding to the target document image is output according to the first candidate text information; If the first candidate text information does not contain any characters that are the same as the target optional characters, the target character that is most similar to the characters included in the first text information is determined from the plurality of target optional characters, and the characters included in the first text information are replaced with the target character, and the recognition result corresponding to the target document image is output.
7. The method according to claim 1, characterized in that, The target recognition template also includes target character rules corresponding to the region to be recognized; the initial recognition result also includes candidate text information corresponding to multiple text regions respectively; The step of outputting the recognition result corresponding to the target document image based on the first text information includes: Detect whether the characters included in the first text information conform to the target character rules; If the characters included in the first text information do not conform to the target character rules, the first candidate text information corresponding to the first text region is obtained from the initial recognition result; Detect whether the characters included in the first candidate text information conform to the target character rules; If the characters included in the first candidate text information conform to the target character rules, the recognition result corresponding to the target document image is output according to the first candidate text information.
8. The method according to claim 1, characterized in that, The step of determining the mapping region location information corresponding to the mapping region of the region to be identified in the target document image based on the mapping relationship between the text region location information corresponding to the target text region and the anchor region location information corresponding to the target anchor region includes: Based on the mapping relationship between the text region location information corresponding to the target text region and the anchor region location information corresponding to the target anchor region, the target recognition template is subjected to perspective transformation according to the target document image to obtain the transformed target recognition template. Based on the transformed target recognition template, the transformed region location information corresponding to the target region location information is determined, which is the mapped region location information corresponding to the mapped region of the region to be recognized in the target document image.
9. A text recognition device, characterized in that, The device includes: The setting module is used to set a target recognition template for the target type of document. The target recognition template includes standard text information and anchor point area location information corresponding to multiple anchor point areas, as well as target area location information corresponding to the area to be recognized. The acquisition module is used to acquire a target document image corresponding to the target document, perform text recognition on the target document image, and obtain an initial recognition result. The initial recognition result includes text information and text region location information corresponding to multiple text regions respectively. The first determining module is used to compare the text information in the initial recognition result with the standard text information in the target recognition template one by one when the type of the target document is the target type, and to determine multiple pairs of target text regions and target anchor point regions that match the text information. The second determining module is used to determine the mapping region position information corresponding to the mapping region of the region to be identified in the target document image based on the mapping relationship between the text region position information corresponding to the target text region and the anchor region position information corresponding to the target anchor region; The third determining module is used to determine, based on the mapping region location information and the text region location information corresponding to the plurality of text regions respectively, a first text region that intersects with the mapping region from the plurality of text regions; The output module is used to obtain first text information corresponding to the first text region from the initial recognition result, and output the recognition result corresponding to the target document image based on the first text information; The target recognition template also includes a text layout format corresponding to the region to be recognized; When there are multiple first text regions, the step of obtaining first text information corresponding to the first text region from the initial recognition result and outputting the recognition result corresponding to the target document image based on the first text information includes: When the text layout is in the form of a single-line text, a first target proportion of the portion where the first text region intersects with the mapped region is determined to be the first target proportion of the mapped region. The text region with the largest proportion of the first target is selected from multiple first text regions and used as the second text region; Obtain the second text information corresponding to the second text region from the initial recognition result; Output the recognition result corresponding to the target document image based on the second text information; The step of obtaining first text information corresponding to the first text region from the initial recognition result and outputting the recognition result corresponding to the target document image based on the first text information includes: When the text layout is in the form of multi-line text, a second target proportion is determined for the portion of the first text region where the first text region intersects with the mapping region. One or more text regions whose second target ratio is greater than a preset threshold are obtained from multiple first text regions and used as second text regions; Obtain the second text information corresponding to the second text region from the initial recognition result; When the number of the second text regions is one, the recognition result corresponding to the target document image is output based on the second text information; When there are multiple second text regions, the second text information corresponding to the multiple text regions is concatenated to obtain third text information, and the recognition result corresponding to the target document image is output based on the third text information.
10. A text recognition device, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the steps of the text recognition method as described in any one of claims 1-8.
11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the steps of the text recognition method as described in any one of claims 1-8.
12. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device causes the electronic device to perform the steps of the text recognition method as described in any one of claims 1-8.