Image recognition method, device, equipment and storage medium

Abstract

The present invention discloses a method, device, equipment and storage medium for image recognition. The method comprises obtaining an image of a document to be recognized, and determining a target recognition template image according to the image of the document to be recognized and a preset pre-classifier; using the target recognition template image to cut out at least one image of a field to be recognized from the image of the document to be recognized; using a preset recognition tool to recognize at least one image of a field to be recognized, and obtaining at least one local recognition result corresponding to at least one image of a field to be recognized; and determining an overall recognition result that matches the image of the document to be recognized according to at least one local recognition result. The technical solution of the present invention greatly improves the efficiency of mixed input of multiple types of documents and improves the feasibility of complex document recognition.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an image recognition method, device, equipment and storage medium. Background Art

[0002] With the development of online government systems, more and more electronic documents need to be entered into government systems for processing. Traditionally, electronic document entry is done manually, and staff need to manually enter the document information into the system, which is time-consuming and laborious.

[0003] Although traditional OCR (Optical Character Recognition) technology is suitable for processing continuous text (such as books), when processing documents with scattered text such as forms or documents, existing text recognition technology has difficulty distinguishing the contents of the document, and thus it is difficult to extract the required information from the document. Although some open source solutions such as PP-Structure that analyze form structures provide new ideas for document recognition, existing solutions are still difficult to produce stable, consistent, and reliable recognition results, and are not suitable for use in government systems.

[0004] In the process of implementing the present invention, the inventors found that the prior art has the following defects:

[0005] Manual entry of electronic documents into government systems is time-consuming and laborious; electronic documents with scattered text are difficult to distinguish and identify. Summary of the invention

[0006] The present invention provides an image recognition method, device, equipment and storage medium to greatly improve the efficiency of mixed input of multiple types of documents and improve the feasibility of complex document recognition.

[0007] According to one aspect of the present invention, a method for image recognition is provided, the method comprising:

[0008] Acquire a document image to be identified, and determine a target identification template image based on the document image to be identified and a preset pre-classifier;

[0009] Using the target recognition template image, cropping at least one field image to be recognized from the document image to be recognized;

[0010] Using a preset recognition tool to recognize the at least one field image to be recognized, and obtaining at least one local recognition result corresponding to the at least one field image to be recognized;

[0011] Based on the at least one local recognition result, an overall recognition result matching the document image to be recognized is determined.

[0012] According to another aspect of the present invention, there is provided a picture recognition device, the device comprising:

[0013] A template image determination module, used to obtain a document image to be identified, and determine a target identification template image based on the document image to be identified and a preset pre-classifier;

[0014] A to-be-recognized field image cropping module, used to crop at least one to-be-recognized field image from the to-be-recognized document image using the target recognition template image;

[0015] A local recognition result acquisition module, used to recognize the at least one field image to be recognized by using a preset recognition tool, and obtain at least one local recognition result corresponding to the at least one field image to be recognized;

[0016] The overall recognition result acquisition module is used to determine the overall recognition result that matches the document image to be recognized based on the at least one local recognition result.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image recognition method described in any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image recognition method described in any embodiment of the present invention when executed.

[0022] The technical solution of the embodiment of the present invention obtains a picture of a document to be identified, and determines a target identification template picture based on the picture of the document to be identified and a preset pre-classifier; uses the target identification template picture to crop at least one field picture to be identified from the picture of the document to be identified; uses a preset identification tool to identify at least one field picture to be identified, and obtains at least one local identification result corresponding to the at least one field picture to be identified; and determines, based on the at least one local identification result, a technical means of an overall identification result that matches the picture of the document to be identified. This solves the problem that the prior art is time-consuming and labor-intensive in manually entering electronic documents into government affairs systems, and that electronic documents with scattered text are difficult to distinguish and identify, greatly improves the efficiency of mixed entry of multiple types of documents, and improves the feasibility of identifying complex documents.

[0023] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1a A schematic diagram of a process of pre-creating a recognition template image provided by an embodiment of the present invention;

[0026] Figure 1b A flowchart of a method for image recognition provided in Embodiment 1 of the present invention;

[0027] Figure 2a A flowchart of another image recognition method provided in Embodiment 2 of the present invention;

[0028] Figure 2b A schematic diagram of the overall process of an image recognition method provided in Embodiment 2 of the present invention;

[0029] Figure 3 A schematic diagram of the structure of an image recognition device provided in Embodiment 3 of the present invention;

[0030] Figure 4 It is a schematic diagram of the structure of an electronic device for implementing the image recognition method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "target", "candidate", "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including", "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] In the present invention, a recognition template image can be created in advance, so that the document image to be recognized can be recognized based on the pre-created recognition template image. Figure 1a A schematic diagram of a process for pre-creating a recognition template image provided by an embodiment of the present invention. For each type of electronic document, a recognition template can be pre-created. A recognition template can include a template image, several positioning fields, several recognition fields, and the classification logic and processing logic of the template.

[0034] Step 1. Template image selection: You can upload a document image as a template image. The template image should be clear and not skewed or distorted (for example, a document image obtained by scanning).

[0035] For the uploaded template image, you can perform the following operations: (1) save the template image; (2) save the height w of the template image; (3) save the width h of the template image.

[0036] Step 2: Create positioning fields: You can select several fields that are inherent to the document on the template image. For example, for a food business license, fields such as "Food Business License", "Business Operator Name", and "License Number" are inherent to the document, which are the positioning fields of the food business license.

[0037] The above positioning fields will be used to detect the position of the document in the document image to be identified when using the recognition template to identify the electronic document image, and accordingly use the homography matrix to fit and align the identification image with the template image to eliminate the distortion of the document in the identification image. Therefore, due to the characteristics of the homography matrix, when selecting the positioning fields, you can try to select one positioning field each near the four corners of the document (that is, select at least four positioning fields, and try to distribute the four fields at the four corners of the document). Selecting more positioning fields can improve the effect of fit and alignment.

[0038] For each selected positioning field, you can perform the following operations:

[0039] (1) Store the coordinates of the upper left corner of the positioning field box P atl (x atl ,y atl );

[0040] (2) Store the coordinates of the lower right corner of the positioning field box P abr (x abr ,y abr );

[0041] (3) Set the positioning field (i.e. template image x atl ≤x≤x abr ,y atl ≤y≤y abr The part) is cut out from the picture to become a separate picture;

[0042] (4) Use a preset recognition tool (such as open source text recognition tools Tesseract and PaddleOCR, etc.) to recognize the cropped image of the positioning field and obtain the characters in the positioning field and the position of each character;

[0043] (5) For each character recognized in the location field, perform the following operations:

[0044] ①Store character content;

[0045] ② The upper left corner coordinate P of the character relative to the positioning field c ' tl (x' ctl , y' ctl ), according to x ctl =x atl +x' ctl ,y ctl =y atl +y' ctl Convert to coordinates P relative to the template image ctl (x ctl ,y ctl ) and store;

[0046] ③ The character is relative to the lower right corner coordinate P of the positioning field c ' br (x' cbr , y' cbr ), according to x cbr =x abr +x' cbr ,y cbr =y abr +y' cbr Convert to coordinates P relative to the template image cbr (x cbr ,y cbr ) and store.

[0047] When creating a location field, you can automatically assign a unique identifier as the name of the location field. You can also customize a field name for each location field to be used as an identifier when writing classification logic later.

[0048] Optionally, the present invention may include the function of editing and deleting the location field. The editing function may include changing the name, position, width, and height of the location field.

[0049] Step 3, create identification fields: You can frame several fields that need to be identified on the template image. For example, for a food business license, if you need to identify the business location, you should completely frame the field corresponding to the business location (XX Road, XX District, XX City, XX Province). If there are multiple fields that need to be identified in the document, you can frame these fields one by one. The identification field does not need to be logically associated with the positioning field.

[0050] For each identified field you select, you can perform the following operations:

[0051] (1) Store the coordinates of the upper left corner of the recognition field box P ftl (x ftl ,y ftl );

[0052] (2) Store the coordinates of the lower right corner of the recognition field box P fbr (x fbr ,y fbr ).

[0053] Optionally, the recognition field may include the attribute of field type. For example, for a purely numeric field, the user may wish that the recognition result does not include non-numeric content. In this case, the field type may be specified as numeric when creating the recognition field, and when using the recognition template to recognize the electronic document, the present invention may traverse the characters in the recognition result and remove non-numeric characters. Possible field types include but are not limited to: general (Chinese, English, numbers, symbols), English and numbers, numbers.

[0054] Optionally, the present invention can configure whether the identification field can be empty. During identification, if the field cannot be found and the configuration of the field can be empty, it is skipped; if it cannot be empty, the corresponding prompt information can be added to the filling test result for review by the staff.

[0055] Optionally, when creating an identification field, a unique identifier can be automatically assigned as the name of the identification field. It is also possible to customize a field name for each identification field for subsequent identification when writing processing logic. In addition, when using an identification template to identify an electronic document, the present invention can output the identification results in pairs in the format of "field name-field identification result", which is convenient for finding the corresponding result by field name after obtaining the identification result.

[0056] Optionally, the present invention may include the function of editing and deleting the identification field. The editing function may include changing the name, position, width, and height of the identification field.

[0057] Step 4: Customization of classification logic function: You can configure a classification logic function def c() for the recognition template to effectively distinguish the documents corresponding to the template during recognition.

[0058] Specifically, a check code can be written for each previously defined positioning field (l1, l2, l3, l4, ...). For example, for field l1, the python example is as follows: m1 = re.match (r'regular expression', l1). If m1 is none, it means that the regular expression match fails. If it is not none, it means that the match is successful. Similarly, similar matching functions can be written for l2, l3, l4, ..., and then the final output of the classification logic function def c() is obtained according to the custom logic (for example, the positioning field matching success rate exceeds the preset threshold). The output should be Boolean. If it is true, it means that the recognition template successfully matches the document image to be recognized.

[0059] Step 5: Customize the processing logic function: You can configure a processing logic def p() for the recognition template to further process the recognized fields. For example, you can process the recognized dates in a unified format of "XX year XX month XX day".

[0060] Specifically, a processing function pi can be written for each previously defined recognition field (r1, r2, r3, r4, ...). For example, for field r1, the python example is as follows: r1 = p1 (r1). After the recognition is completed, def p() is executed once for the recognized field (r1, r2, r3, r4, ...), thereby updating the processed recognition field set.

[0061] Step 6. Update the pre-classifier: When creating a recognition template, the appearance features of the template (color, lines, layout, etc.) will be extracted and updated to the pre-classifier. When performing recognition, the pre-classifier can be used to preliminarily identify the recognition template with high appearance similarity, and then the classification logic function can be used for accurate matching.

[0062] Embodiment 1

[0063] Figure 1b This is a flow chart of an image recognition method provided in the first embodiment of the present invention. This embodiment can be applied to the case where electronic document images uploaded to the government affairs system are recognized. The method can be executed by an image recognition device, which can be implemented in the form of hardware and / or software. The image recognition device can be configured in a server with image processing function. Figure 1b As shown, the method includes:

[0064] S110, obtaining a document image to be identified, and determining a target identification template image according to the document image to be identified and a preset pre-classifier.

[0065] The document image to be identified may refer to an image of an electronic document whose text needs to be identified, such as a food business license. The preset pre-classifier may be used to determine the attribute category of the document image to be identified, and to determine a matching recognition template for the document image to be identified. The target recognition template image may be a template image selected from a plurality of pre-created recognition template images that matches the document image to be identified.

[0066] In this embodiment, a preset pre-classifier may be used to determine a recognition template image that matches the document image to be recognized in a pre-created recognition template image library.

[0067] In this embodiment, by adding a preset pre-classifier in the recognition process, the recognition template image can be automatically matched with the document image to be recognized, thereby improving the recognition efficiency.

[0068] In an optional embodiment, determining a target recognition template image based on the document image to be recognized and a preset pre-classifier may include: determining a set of candidate recognition template images that match the document image to be recognized based on the document image to be recognized and a preset pre-classifier; and matching each candidate recognition template image in the set of candidate recognition template images with the document image to be recognized to determine the target recognition template image.

[0069] On the basis of the above optional implementation mode, according to the document image to be identified and the preset pre-classifier, a set of candidate identification template images that match the document image to be identified is determined. The preset pre-classifier can be used to screen out candidate identification template images whose similarity with the document image to be identified exceeds a preset similarity threshold from the preset identification template image library; the candidate identification template images are arranged in a preset order to obtain a set of candidate identification template images.

[0070] The preset order may refer to an order from high to low similarity, or an order from low to high similarity.

[0071] On the basis of the above optional implementation mode, the step of determining the target recognition template image may include: determining the current candidate recognition template image in the candidate recognition template image set in sequence; obtaining the first type of positioning field area in the current candidate recognition template image, and obtaining the second type of positioning field area of ​​the document image to be recognized; using the second type of positioning field area as a reference, proportionally adjusting the first type of positioning field area to obtain the adjusted first type of positioning field area; using the classification logic function to check the adjusted first type of positioning field area, and judging whether the current candidate recognition template image and the document image to be recognized are matched successfully according to the output result of the classification logic function; when the match is successful, determining the current candidate recognition template image as the target recognition template image; when the match is unsuccessful, judging whether all candidate recognition template images are matched with the document image to be recognized; when the match is completed, generating a first prompt message to prompt manual determination of the target recognition template image; when the match is not completed, returning to execute the operation of determining the current candidate recognition template image.

[0072] In this embodiment, the candidate recognition template images with high similarity to the document image to be recognized can be preferentially used as the current candidate recognition template images, that is, the current candidate recognition template images are determined in descending order of similarity.

[0073] Specifically, for each positioning field in the current candidate recognition template image, find the position of each character in this field in the document image to be recognized. This can be divided into the following steps:

[0074] (1) Considering that the size and aspect ratio of the document image to be identified may be inconsistent with those of the current candidate recognition template image, the positioning field box of the current candidate recognition template image (i.e., the first type of positioning field area) can be proportionally converted to the positioning field box of the document image to be identified (i.e., the second type of positioning field area). Assuming that the width of the document image to be identified is w' and the height is h', then for any point P(x, y) in the current candidate recognition template image, its position in the document image to be identified is Q(x÷w×w', y÷h×h'), where w represents the width of the current candidate recognition template image and h represents the height of the current candidate recognition template image. Therefore, in this step, the upper left corner P of the positioning field box of the current candidate recognition template image is converted to the position of the document image to be identified. atl , bottom right corner P abr The coordinates are converted to the upper left corner Q of the positioning field box of the document image to be identified atl , Q in the lower right corner abr coordinate.

[0075] (2) Since the document in the document image to be identified may be skewed or distorted, the corresponding positioning field may not be completely included in the converted positioning field area (i.e., the upper left corner Q atl , Q in the lower right corner abr Therefore, it is necessary to expand the converted positioning field area. For example, expand the length and width of the area to 2 times. Get the expanded upper left corner Q' atl , lower right corner Q' abr coordinate.

[0076] In the above description, the upper left corner Q' atl and the lower right corner Q' abr The area represented is the first type of positioning field area after the above adjustment.

[0077] Furthermore, the classification logic function can be used to check the adjusted first-category positioning field area, and determine whether the current candidate recognition template image and the document image to be recognized are successfully matched according to the output result of the classification logic function. For example, the positioning field of a first-category positioning field area after adjustment is represented as l1, and the corresponding logic function m1=re.match(r'regular expression', l1) is used. If m1 is none, it means that the regular expression fails to match, and if it is not none, it means that the match is successful.

[0078] When the match is successful, the current candidate recognition template image may be determined as the target recognition template image.

[0079] When the match is unsuccessful, it can be determined whether all candidate recognition template images have been matched with the document image to be recognized; when the match is completed, a first prompt message is generated to prompt manual determination of the target recognition template image; when the match is not completed, the operation of determining the current candidate recognition template image is returned.

[0080] In this embodiment, the classification logic function is added to more accurately classify the document image to be identified, and more accurately determine the target recognition template image for the document image to be identified, thereby achieving more accurate identification.

[0081] S120: Using the target recognition template image, cropping at least one field image to be recognized from the document image to be recognized.

[0082] Optionally, before using the target recognition template image to crop at least one field image to be recognized from the document image to be recognized, the target recognition template image and the document image to be recognized may be aligned. In this embodiment, the document in the document image to be recognized may be transformed in perspective through a homography matrix to achieve an effect of completely coinciding with the document in the target recognition template image. The specific steps may be as follows:

[0083] Step 1: Use the coordinates in the matching result list of the target recognition template image and the document image to be recognized to calculate the homography matrix H from the document image to be recognized to the target recognition template image. Then, any point Q (x', y') in the document image to be recognized and its corresponding point P (x, y) on the target recognition template image have the following relationship: in,

[0084] Step 2: Use the pairwise corresponding coordinates in the matching result list to estimate H based on the least squares method, that is, minimize:

[0085] Step 3: When the matching results are sufficient, the estimation based on the least squares method can eliminate or reduce the possible errors in the identification and matching of the positioning points. 33 is a scaling parameter, so it can be normalized to 1.

[0086] Step 4: Use the estimated homography matrix to align the document image to be identified and the target recognition template image. First, create a blank new image with the same size as the target recognition template image (i.e., width w, length h), then traverse each point in the document image to be identified, and set the pixel value of the corresponding point on the blank new image to the pixel value of the point (i.e., for point Q(x', y') in the document image to be identified, set the point of the blank new image to the pixel value of the point). The new image finally obtained is the document image to be identified that is aligned with the target recognition template image.

[0087] Optionally, in step 3, considering that the identification and matching of the positioning points may contain outliers due to recognition errors, robust algorithms such as RANSAC and RHO can be introduced to automatically identify and eliminate outliers in the matching results, thereby further improving the accuracy of homography matrix estimation.

[0088] Optionally, in order to avoid sampling distortion in the new image, the transformed source image and target image can be inverted, that is, the homography matrix from the target recognition template image to the document image to be recognized is estimated in step 3, and the pixels of the new image are traversed in step 4.

[0089] Furthermore, since the document in the new image has been aligned with the document in the target recognition template image, the recognition field area in the target recognition template image is the field area to be recognized in the new image. By simply cropping the field to be recognized from the new image, at least one field to be recognized image of the document to be recognized can be obtained.

[0090] S130: Use a preset recognition tool to recognize at least one field image to be recognized, and obtain at least one local recognition result corresponding to the at least one field image to be recognized.

[0091] The preset recognition tool may be, for example, open source text recognition tools such as Tesseract and PaddleOCR.

[0092] In this embodiment, a preset recognition tool is used to recognize the characters in each field image to be recognized, and a corresponding recognition result is obtained. The recognition result of each field image to be recognized is a local recognition result relative to the entire document image to be recognized.

[0093] S140. Determine an overall recognition result that matches the document image to be recognized based on at least one local recognition result.

[0094] In this embodiment, the overall recognition result can be determined based on at least one local recognition result of the document image to be recognized.

[0095] The technical solution of the embodiment of the present invention obtains a picture of a document to be identified, and determines a target identification template picture based on the picture of the document to be identified and a preset pre-classifier; uses the target identification template picture to crop at least one field picture to be identified from the picture of the document to be identified; uses a preset identification tool to identify at least one field picture to be identified, and obtains at least one local identification result corresponding to the at least one field picture to be identified; and determines, based on the at least one local identification result, a technical means of an overall identification result that matches the picture of the document to be identified. This solves the problem that the prior art is time-consuming and labor-intensive in manually entering electronic documents into government affairs systems, and that electronic documents with scattered text are difficult to distinguish and identify, greatly improves the efficiency of mixed entry of multiple types of documents, and improves the feasibility of identifying complex documents.

[0096] Embodiment 2

[0097] Figure 2a This is a flowchart of another image recognition method provided in the second embodiment of the present invention. Based on the above embodiment, this embodiment refines the two operations of using the target recognition template image to crop the field image to be recognized from the document image to be recognized and determining the overall recognition result that matches the document image to be recognized based on at least one local recognition result. Figure 2a As shown, the method includes:

[0098] S210, obtaining a document image to be identified, and determining a target identification template image according to the document image to be identified and a preset pre-classifier.

[0099] S220, using the target recognition template image to determine the field area to be recognized from the document image to be recognized.

[0100] S230, cropping the image of the field to be identified from the image of the document to be identified according to the area of ​​the field to be identified.

[0101] S240: Use a preset recognition tool to recognize at least one field image to be recognized, and obtain at least one local recognition result corresponding to the at least one field image to be recognized.

[0102] In this embodiment, an open source text recognition tool (such as Tesseract, PaddleOCR) may be used to recognize the image of the field to be recognized.

[0103] Optionally, the image of the field to be identified can be preprocessed to assist the text recognition tool in recognition. Possible processing methods may include but are not limited to binarization, noise removal, corrosion, expansion, opening operation, and closing operation. Preprocessing makes the text clearer and easier to recognize.

[0104] Optionally, if the field to be identified in the field to be identified image has a field type, the output of the text recognition tool can be filtered according to the field type. For example, if the field type of the field to be identified is numeric, non-numeric text in the output is removed.

[0105] S250. Perform an abnormal completion probability test on at least one local recognition result using a pre-trained anomaly detection model to obtain at least one probability detection result.

[0106] Among them, the anomaly detection model can be obtained by training a natural language processing model. Specifically, the anomaly detection model can be obtained by collecting historical abnormal document recognition data for manual annotation, and then training the natural language processing model with the abnormal corpus data to obtain the anomaly detection model, so that the anomaly detection model can identify whether there is intentional wrong filling or random filling based on the input corpus. The probability detection result can refer to the abnormal filling probability value of the local recognition result.

[0107] In this embodiment, after the image of the field to be identified is identified, a local recognition result is obtained, the local recognition result is input into the anomaly detection model, and the abnormal filling probability value of each local recognition result is output.

[0108] Optionally, after obtaining at least one probability detection result, it may also include: comparing each probability detection result with a preset probability threshold; determining the local recognition result corresponding to the probability detection result exceeding the preset probability threshold as an abnormal local recognition result; and generating a second prompt information to prompt manual verification of the abnormal local recognition result.

[0109] Among them, the preset probability threshold can be the upper limit value of the abnormal filling probability of the local recognition result, that is, after the probability detection result exceeds the preset probability threshold, it is determined that the current local recognition result is obviously abnormal.

[0110] Specifically, the probability detection result of each local recognition result needs to be compared with a preset probability threshold. A local recognition result that exceeds the preset probability threshold can be determined as an abnormal local recognition result, and a second prompt information is generated for the abnormal local recognition result to prompt manual verification of the abnormal local recognition result.

[0111] S260. When it is determined that at least one local recognition result is normally filled in according to at least one probability detection result and a preset probability threshold, at least one local recognition result is updated according to the required format using a preset processing logic function to obtain at least one updated local recognition result.

[0112] In this embodiment, if it is determined that each probability detection result does not exceed the preset probability threshold, it is determined that each local recognition result is filled in normally, and the preset processing logic function configured in the target recognition template image can be used to update the local recognition results that need to be uniformly processed according to the required format, thereby obtaining the updated local recognition results.

[0113] For example, the processing logic function def p() is executed for the local recognition results (r1, r2, r3, r4, ...) to update the set (r1, r2, r3, r4, ...).

[0114] The advantage of this setting is that by adding processing logic functions, the automated processing of specified identification fields can be achieved.

[0115] S270: Merge at least one updated local recognition result to obtain an overall recognition result.

[0116] In this embodiment, the local recognition results can be merged and processed into structured data (such as JSON, XML, YAML) output to obtain the overall recognition result.

[0117] For example, Figure 2b A schematic diagram of the overall process of an image recognition method provided for the second embodiment of the present invention. A candidate recognition template image set of the document image to be recognized is found by a preset pre-classifier; the current candidate recognition image template is determined, and the positioning field of the current candidate recognition template image is extracted; the positioning field of the current candidate recognition template image and the positioning field of the document image to be recognized are matched using classification logic, and when the match is successful, the target recognition template image is determined, the document image to be recognized and the target recognition template image are aligned, and the field to be recognized of the document image to be recognized is cropped, so that the field to be recognized is recognized, and a local recognition result is obtained, and the local recognition result is filled in and checked, and when it is determined that the filling is normal, the local recognition result is processed according to the processing logic function (when the filling is abnormal, it can be prompted for manual verification), and the processed local recognition results are merged to obtain the overall recognition result; when the match is unsuccessful, it is determined whether all candidate recognition template images have been processed, and if so, it is prompted to manually select the target recognition template image, otherwise the current candidate recognition image template is re-determined.

[0118] In summary, the technical solution of the embodiment of the present invention proposes a template matching method that combines a pre-classifier and classification logic. Through the classification logic of the template configuration, the candidate templates preliminarily screened out by the pre-classifier are confirmed, and finally the target template that matches the document image to be identified is found to realize automatic selection of the identification template; by adding a custom processing logic function to the identification template, the automatic processing of the specified identification field is realized; the filling inspection method based on natural language processing trains the model through the abnormal corpus data set, and realizes the identification of incorrect filling, random filling and other behaviors, and reminds the staff to pay attention.

[0119] The technical solution of the embodiment of the present invention obtains a picture of a document to be identified, determines a target identification template picture according to the picture of the document to be identified and a preset pre-classifier; determines the field area to be identified by using the target identification template picture; crops the field picture to be identified according to the field area to be identified; identifies at least one field picture to be identified by using a preset identification tool to obtain at least one local identification result; performs abnormal filling probability detection on at least one local identification result by using a pre-trained anomaly detection model to obtain at least one probability detection result; when it is determined that at least one local identification result is filled in normally according to at least one probability detection result and a preset probability threshold, updates at least one local identification result according to the required format by using a preset processing logic function to obtain at least one updated local identification result; and merges at least one updated local identification result to obtain a technical means for the overall identification result, which solves the problems of the prior art that it is time-consuming and labor-intensive to manually enter electronic documents in government affairs systems and that it is difficult to distinguish and identify electronic documents with scattered text, greatly improves the efficiency of mixed entry of multiple types of documents, and improves the feasibility of complex document recognition.

[0120] Embodiment 3

[0121] Figure 3 This is a schematic diagram of the structure of an image recognition device provided by Embodiment 3 of the present invention. Figure 3 As shown, the device includes: a template image determination module 310, a to-be-recognized field image cropping module 320, a local recognition result acquisition module 330 and an overall recognition result acquisition module 340. Among them:

[0122] The template image determination module 310 is used to obtain a document image to be identified, and determine a target identification template image according to the document image to be identified and a preset pre-classifier;

[0123] A to-be-recognized field image cropping module 320, configured to crop at least one to-be-recognized field image from the to-be-recognized document image using the target recognition template image;

[0124] The local recognition result acquisition module 330 is used to recognize the at least one field image to be recognized by using a preset recognition tool, and obtain at least one local recognition result corresponding to the at least one field image to be recognized;

[0125] The overall recognition result acquisition module 340 is used to determine the overall recognition result that matches the document image to be recognized based on the at least one local recognition result.

[0126] The technical solution of the embodiment of the present invention obtains a picture of a document to be identified, and determines a target identification template picture based on the picture of the document to be identified and a preset pre-classifier; uses the target identification template picture to crop at least one field picture to be identified from the picture of the document to be identified; uses a preset identification tool to identify at least one field picture to be identified, and obtains at least one local identification result corresponding to the at least one field picture to be identified; and determines, based on the at least one local identification result, a technical means of an overall identification result that matches the picture of the document to be identified. This solves the problem that the prior art is time-consuming and labor-intensive in manually entering electronic documents into government affairs systems, and that electronic documents with scattered text are difficult to distinguish and identify, greatly improves the efficiency of mixed entry of multiple types of documents, and improves the feasibility of identifying complex documents.

[0127] Optionally, the template image determination module 310 includes:

[0128] A candidate recognition template picture set determination unit, used to determine a candidate recognition template picture set matching the document picture to be recognized based on the document picture to be recognized and a preset pre-classifier;

[0129] The target recognition template image determination unit is used to match each candidate recognition template image in the candidate recognition template image set with the document image to be recognized, so as to determine the target recognition template image.

[0130] Optionally, the candidate recognition template picture set determination unit may be specifically used to:

[0131] Using the preset front classifier, a candidate recognition template image whose similarity with the document image to be recognized exceeds a preset similarity threshold is screened out from a preset recognition template image library;

[0132] The candidate recognition template images are arranged in a preset order to obtain a candidate recognition template image set.

[0133] Optionally, the target recognition template image determination unit may be used to:

[0134] Determine the current candidate recognition template picture in the candidate recognition template picture set in sequence;

[0135] Acquire the first type of positioning field area in the current candidate recognition template image, and acquire the second type of positioning field area in the document image to be recognized;

[0136] Taking the second type of positioning field area as a reference, proportionally adjusting the first type of positioning field area to obtain an adjusted first type of positioning field area;

[0137] Using a preset classification logic function to check the adjusted first-category positioning field area, and judging whether the current candidate recognition template image and the document image to be recognized are matched successfully according to the output result of the classification logic function;

[0138] If so, determining the current candidate recognition template image as the target recognition template image;

[0139] Otherwise, determine whether all candidate recognition template images have been matched with the document image to be recognized; if so, generate a first prompt message to prompt manual determination of the target recognition template image; otherwise, return to execute the operation of determining the current candidate recognition template image.

[0140] Optionally, the to-be-identified field image cropping module 320 may be specifically used for:

[0141] Determine the field area to be identified from the document image to be identified using the target identification template image;

[0142] The image of the field to be identified is cropped from the image of the document to be identified according to the area of ​​the field to be identified.

[0143] Optionally, the overall recognition result acquisition module 340 may be specifically used for:

[0144] Using a pre-trained anomaly detection model to perform an abnormal filling probability detection on the at least one local recognition result, and obtain at least one probability detection result;

[0145] When it is determined according to the at least one probability detection result and the preset probability threshold that the at least one local recognition result is filled in normally, the at least one local recognition result is updated according to the required format by using the preset processing logic function to obtain the at least one updated local recognition result;

[0146] The updated at least one local recognition result is merged to obtain the overall recognition result.

[0147] Optionally, the image recognition device further includes a prompt verification module, which is used to, after obtaining at least one probability detection result:

[0148] Compare each probability detection result with the preset probability threshold value respectively;

[0149] Determining the local recognition result corresponding to the probability detection result exceeding the preset probability threshold as an abnormal local recognition result;

[0150] A second prompt message is generated to prompt manual verification of the abnormal local recognition result.

[0151] The image recognition device provided in the embodiment of the present invention can execute the image recognition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0152] Embodiment 4

[0153] Figure 4 A schematic diagram of the structure of an electronic device 400 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0154] like Figure 4 As shown, the electronic device 400 includes at least one processor 401, and a memory connected to the at least one processor 401 in communication, such as a read-only memory (ROM) 402, a random access memory (RAM) 403, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 401 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 402 or the computer program loaded from the storage unit 408 to the random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 can also be stored. The processor 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0155] Multiple components in the electronic device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the electronic device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0156] Processor 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of processor 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. Processor 401 performs the various methods and processes described above, such as a picture recognition method.

[0157] In some embodiments, the image recognition method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 400 via the ROM 402 and / or the communication unit 409. When the computer program is loaded into the RAM 403 and executed by the processor 401, one or more steps of the image recognition method described above may be performed. Alternatively, in other embodiments, the processor 401 may be configured to perform the image recognition method in any other appropriate manner (e.g., by means of firmware).

[0158] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0159] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0160] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0161] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0162] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0163] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0164] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0165] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for image recognition, characterized in that: include: Acquire a document image to be identified, and determine a target identification template image based on the document image to be identified and a preset pre-classifier; Using the target recognition template image, cropping at least one field image to be recognized from the document image to be recognized; Using a preset recognition tool to recognize the at least one field image to be recognized, and obtaining at least one local recognition result corresponding to the at least one field image to be recognized; Determining an overall recognition result that matches the document image to be recognized based on the at least one local recognition result; The step of determining a target recognition template image based on the document image to be recognized and a preset pre-classifier includes: Determine a candidate recognition template image set matching the document image to be recognized based on the document image to be recognized and a preset pre-classifier; Matching each candidate recognition template image in the candidate recognition template image set with the document image to be recognized to determine the target recognition template image; The step of matching each candidate recognition template image in the candidate recognition template image set with the document image to be recognized to determine the target recognition template image includes: Determine the current candidate recognition template picture in the candidate recognition template picture set in sequence; Acquire the first type of positioning field area in the current candidate recognition template image, and acquire the second type of positioning field area in the document image to be recognized; Taking the second type of positioning field area as a reference, proportionally adjusting the first type of positioning field area to obtain an adjusted first type of positioning field area; Using a preset classification logic function to check the adjusted first-category positioning field area, and judging whether the current candidate recognition template image and the document image to be recognized are matched successfully according to the output result of the classification logic function; If so, determining the current candidate recognition template image as the target recognition template image; Otherwise, determine whether all candidate recognition template images have been matched with the document image to be recognized; if so, generate a first prompt message to prompt manual determination of the target recognition template image; otherwise, return to execute the operation of determining the current candidate recognition template image.

2. The method according to claim 1, characterized in that Determining a candidate recognition template picture set matching the document picture to be recognized according to the document picture to be recognized and a preset pre-classifier, including: Using the preset front classifier, a candidate recognition template image whose similarity with the document image to be recognized exceeds a preset similarity threshold is screened out from a preset recognition template image library; The candidate recognition template images are arranged in a preset order to obtain a candidate recognition template image set.

3. The method according to claim 1, characterized in that Using the target recognition template image to crop the field image to be recognized from the document image to be recognized, comprising: Determine the field area to be identified from the document image to be identified using the target identification template image; The image of the field to be identified is cropped from the image of the document to be identified according to the area of ​​the field to be identified.

4. The method according to claim 1, characterized in that Determining an overall recognition result that matches the document image to be recognized based on the at least one local recognition result includes: Using a pre-trained anomaly detection model to perform an abnormal filling probability detection on the at least one local recognition result, and obtain at least one probability detection result; When it is determined according to the at least one probability detection result and the preset probability threshold that the at least one local recognition result is filled in normally, the at least one local recognition result is updated according to the required format by using the preset processing logic function to obtain the at least one updated local recognition result; The updated at least one local recognition result is merged to obtain the overall recognition result.

5. The method according to claim 4, characterized in that After obtaining at least one probability detection result, the method further includes: Compare each probability detection result with the preset probability threshold value respectively; Determining the local recognition result corresponding to the probability detection result exceeding the preset probability threshold as an abnormal local recognition result; A second prompt message is generated to prompt manual verification of the abnormal local recognition result.

6. A picture recognition device, characterized in that: include: A template image determination module is used to obtain a document image to be identified, and determine a target identification template image based on the document image to be identified and a preset pre-classifier; A to-be-recognized field image cropping module, used to crop at least one to-be-recognized field image from the to-be-recognized document image using the target recognition template image; A local recognition result acquisition module, used to recognize the at least one field image to be recognized by using a preset recognition tool, and obtain at least one local recognition result corresponding to the at least one field image to be recognized; An overall recognition result acquisition module, used to determine an overall recognition result matching the document image to be recognized based on the at least one local recognition result; Wherein, the template image determination module includes: A candidate recognition template picture set determination unit, used to determine a candidate recognition template picture set matching the document picture to be recognized based on the document picture to be recognized and a preset pre-classifier; A target recognition template image determination unit, used to match each candidate recognition template image in the candidate recognition template image set with the document image to be recognized, and determine the target recognition template image; Among them, the target recognition template image determination unit is specifically used to determine the current candidate recognition template image in the candidate recognition template image set in sequence; obtain the first type of positioning field area in the current candidate recognition template image, and obtain the second type of positioning field area of ​​the document image to be recognized; with the second type of positioning field area as a reference, the first type of positioning field area is proportionally adjusted to obtain the adjusted first type of positioning field area; the adjusted first type of positioning field area is checked using a preset classification logic function, and whether the current candidate recognition template image and the document image to be recognized are successfully matched according to the output result of the classification logic function; if so, the current candidate recognition template image is determined as the target recognition template image; otherwise, it is determined whether all candidate recognition template images are matched with the document image to be recognized; if so, a first prompt message is generated to prompt manual determination of the target recognition template image; otherwise, return to execute the operation of determining the current candidate recognition template image.

7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image recognition method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image recognition method according to any one of claims 1 to 5 when executed.

Images