A certificate recognition method, device, equipment and storage medium

Abstract

The present invention provides a document recognition method, device, equipment and storage medium, the method comprising obtaining a picture of a document to be recognized, performing target detection on the picture of the document to be recognized, determining a document area of ​​the document to be recognized and keywords in the document area, and determining the position of the keywords, processing the position of the keywords using preset rules, determining a text area corresponding to the keywords, recognizing the text area corresponding to the keywords, and determining the text content corresponding to the text area. By putting target detection and preset rules together to replace the original document detection and text detection, the document area of ​​the document to be recognized and the keywords in the document area can be detected quickly and accurately at the same time, and the text area corresponding to the keywords is framed using preset rules, which can help improve the performance of document recognition and has strong versatility for the recognition of the same type of documents.

Description

Technical Field

[0001] The present invention relates to the field of financial technology (Fintech), and in particular to a document recognition method, device, equipment and storage medium. Background Art

[0002] With the development of computer technology, more and more technologies are applied in the financial field. The traditional financial industry is gradually transforming to financial technology. However, due to the security and real-time requirements of the financial industry, higher requirements are also placed on technology. For example, when customers handle financial business, in order to ensure the security of business operations, it is necessary to verify the customer's identity information. Generally, customers upload their ID photos in the customer business processing system so that business personnel can use OCR (Optical Character Recognition) technology to verify the ID photos.

[0003] Existing ID recognition methods generally use OCR technology to perform ID detection (detecting the ID area), text detection (detecting the ID text area) and text recognition on ID photos uploaded by users, and identify the text information (such as name, ID number, etc.) in the ID photos uploaded by users for verification by business personnel. However, this processing method requires performing text detection after performing ID detection on the ID photos uploaded by users, resulting in a long ID photo detection process and low performance. Among them, because text detection requires a large amount of text annotation and it is difficult to obtain ID samples, the cost of text annotation is relatively high. In addition, the results of text detection are also greatly affected by the layout of the ID and the density of text.

[0004] In summary, there is an urgent need for a document recognition method to improve the accuracy and efficiency of document recognition. Summary of the invention

[0005] The present invention provides a document recognition method, device, equipment and storage medium for improving the accuracy and efficiency of document recognition.

[0006] In a first aspect, the present invention provides a document recognition method, comprising:

[0007] Get the image of the document to be identified;

[0008] Performing target detection on the image of the document to be identified, determining the document area of ​​the document to be identified and the keywords in the document area, and determining the positions of the keywords;

[0009] Processing the position of the keyword using a preset rule to determine the text area corresponding to the keyword;

[0010] The text area corresponding to the keyword is identified to determine the text content corresponding to the text area.

[0011] In the above technical scheme, by performing target detection on the image of the certificate to be identified, the certificate area of ​​the certificate to be identified and the keywords in the certificate area can be detected quickly and accurately at the same time, and based on the certificate area of ​​the certificate to be identified and the keywords in the certificate area, the position of the keyword can be accurately determined, and the position of the keyword is processed using a preset rule to determine the text area corresponding to the keyword. After that, the text area corresponding to the keyword is identified, and the text content corresponding to the text area can be accurately and quickly determined. Since the existing certificate recognition method is to perform certificate detection on the certificate to be identified and then perform text detection, resulting in a long process and low performance for the certificate detection to be identified, by putting target detection and preset rules together to replace the original certificate detection and text detection, the certificate area of ​​the certificate to be identified and the keywords in the certificate area can be detected quickly and accurately at the same time, and the text area corresponding to the keyword is framed using the preset rules, which can help improve the performance of certificate recognition and has strong versatility for the recognition of the same type of certificates.

[0012] Optionally, performing target detection on the image of the document to be identified, determining a document area of ​​the document to be identified and keywords in the document area, and determining positions of the keywords includes:

[0013] Inputting the image of the document to be identified into a multi-target detection model for identification, and determining the document area of ​​the document to be identified and the keywords in the document area; the multi-target detection model is determined by training a preset instance segmentation model using the first document image sample set;

[0014] Based on the document area of ​​the document to be identified and the keyword in the document area, the position of the keyword is determined.

[0015] In the above technical solution, by inputting the image of the document to be identified into the multi-target detection model for identification, the document area of ​​the document to be identified and the keywords in the document area can be detected quickly and accurately at the same time, and the location of the keywords can be directly and quickly determined. And because the features of the document to be identified are obvious, only a small amount of training samples are needed when training the preset instance segmentation model to enable the multi-target detection model to achieve a good detection effect, which is very practical for documents that are difficult to obtain training samples.

[0016] Optionally, the using a preset rule to process the position of the keyword to determine the text area corresponding to the keyword includes:

[0017] Based on the position of the keyword, the area where the keyword is located is scaled according to a preset scaling ratio to determine the text area corresponding to the keyword.

[0018] In the above technical solution, by scaling the area where the keyword is located according to a certain scaling ratio based on the keyword's position, the text area corresponding to the keyword's position can be directly and effectively framed.

[0019] Optionally, after determining the text area corresponding to the keyword, the method further includes:

[0020] Determine whether there are multiple lines or columns of text in the text area corresponding to the keyword;

[0021] If so, the text area corresponding to the keyword is segmented using a preset segmentation rule to determine multiple text areas corresponding to the keyword; otherwise, a single row or single column text area corresponding to the keyword is determined as the text area corresponding to the keyword.

[0022] In the above technical solution, after determining the text area corresponding to the keyword, it is necessary to determine whether the text area corresponding to the keyword has multiple lines or columns of text, and when it is determined that the text area corresponding to the keyword has multiple lines or columns of text, use the preset cutting rules to cut the text area corresponding to the keyword, so as to provide support for the subsequent text recognition model to accurately identify the text content corresponding to the text area.

[0023] Optionally, the using a preset cutting rule to cut the text area corresponding to the keyword to determine a plurality of text areas corresponding to the keyword includes:

[0024] Taking a screenshot of the text area corresponding to the keyword to obtain a picture of the text area corresponding to the keyword;

[0025] Grayscale processing is performed on the text area image corresponding to the keyword to obtain a blank part, and the blank part is cut to obtain multiple text areas corresponding to the keyword.

[0026] In the above technical solution, by graying the text area image corresponding to the keyword, the blank part can be accurately obtained, and by cutting the blank part, multiple text areas corresponding to the keyword can be accurately obtained, so as to provide support for the subsequent text recognition model to accurately identify the text content corresponding to the text area.

[0027] Optionally, the using the first document image sample set to train a preset instance segmentation model to determine the multi-target detection model includes:

[0028] Obtaining the first certificate image sample set;

[0029] Annotating keywords in the first ID image sample set to obtain an annotated first ID image sample set;

[0030] Performing vectorization processing on the labeled first ID image sample set to obtain a sample vector set corresponding to the labeled first ID image sample set;

[0031] The sample vector set corresponding to the labeled first certificate image sample set is input into the preset instance segmentation model for training to obtain the multi-target detection model.

[0032] In the above technical scheme, by vectorizing the labeled first ID card image sample set, a sample vector set corresponding to the labeled first ID card image sample set is obtained, and the sample vector set corresponding to the labeled first ID card image sample set is input into a preset instance segmentation model for training to obtain a multi-target detection model, which can provide support for quickly and accurately detecting the ID card area of ​​the ID card to be identified and the keywords in the ID card area at the same time.

[0033] Optionally, before identifying the text area corresponding to the keyword, the method further includes:

[0034] Obtain a second document image sample set;

[0035] Annotating text content in the second ID image sample set to obtain an annotated second ID image sample set;

[0036] Performing vectorization processing on the labeled second ID image sample set to obtain a sample vector set corresponding to the labeled second ID image sample set;

[0037] Inputting the sample vector set corresponding to the annotated second certificate image sample set into a preset text recognition model for training to obtain a text recognition model;

[0038] The step of identifying the text area corresponding to the keyword and determining the text content corresponding to the text area includes:

[0039] The text area corresponding to the keyword is input into the text recognition model for recognition, and the text content corresponding to the text area is determined.

[0040] In the above technical solution, by vectorizing the labeled second ID image sample set, a sample vector set corresponding to the labeled second ID image sample set is obtained, and the sample vector set corresponding to the labeled second ID image sample set is input into a preset text recognition model for training to obtain a text recognition model, which can provide support for accurately identifying the text content corresponding to the text area. After that, the text area corresponding to the keyword is input into the text recognition model for recognition, and the text content corresponding to the text area can be quickly and accurately determined.

[0041] In a second aspect, the present invention further provides a document recognition device, comprising:

[0042] An acquisition unit, used to acquire a picture of a document to be identified;

[0043] A processing unit is used to perform target detection on the image of the document to be identified, determine the document area of ​​the document to be identified and the keywords in the document area, and determine the position of the keywords; use preset rules to process the position of the keywords to determine the text area corresponding to the keywords; identify the text area corresponding to the keywords, and determine the text content corresponding to the text area.

[0044] Optionally, the processing unit is specifically configured to:

[0045] Inputting the image of the document to be identified into a multi-target detection model for identification, and determining the document area of ​​the document to be identified and the keywords in the document area; the multi-target detection model is determined by training a preset instance segmentation model using the first document image sample set;

[0046] Based on the document area of ​​the document to be identified and the keyword in the document area, the position of the keyword is determined.

[0047] Optionally, the processing unit is specifically configured to:

[0048] Based on the position of the keyword, the area where the keyword is located is scaled according to a preset scaling ratio to determine the text area corresponding to the keyword.

[0049] Optionally, the processing unit is further configured to:

[0050] After determining the text area corresponding to the keyword, determining whether there are multiple rows or columns of text in the text area corresponding to the keyword;

[0051] If so, the text area corresponding to the keyword is segmented using a preset segmentation rule to determine multiple text areas corresponding to the keyword; otherwise, a single row or single column text area corresponding to the keyword is determined as the text area corresponding to the keyword.

[0052] Optionally, the processing unit is specifically configured to:

[0053] Taking a screenshot of the text area corresponding to the keyword to obtain a picture of the text area corresponding to the keyword;

[0054] Grayscale processing is performed on the text area image corresponding to the keyword to obtain a blank part, and the blank part is cut to obtain multiple text areas corresponding to the keyword.

[0055] Optionally, the processing unit is specifically configured to:

[0056] Obtaining the first certificate image sample set;

[0057] Annotating keywords in the first ID image sample set to obtain an annotated first ID image sample set;

[0058] Performing vectorization processing on the labeled first ID image sample set to obtain a sample vector set corresponding to the labeled first ID image sample set;

[0059] The sample vector set corresponding to the labeled first certificate image sample set is input into the preset instance segmentation model for training to obtain the multi-target detection model.

[0060] Optionally, the processing unit is further configured to:

[0061] Before identifying the text area corresponding to the keyword, obtaining a second document image sample set;

[0062] Annotating text content in the second ID image sample set to obtain an annotated second ID image sample set;

[0063] Performing vectorization processing on the labeled second ID image sample set to obtain a sample vector set corresponding to the labeled second ID image sample set;

[0064] Inputting the sample vector set corresponding to the annotated second certificate image sample set into a preset text recognition model for training to obtain a text recognition model;

[0065] The processing unit is specifically used for:

[0066] The text area corresponding to the keyword is input into the text recognition model for recognition, and the text content corresponding to the text area is determined.

[0067] In a third aspect, the present invention provides a computing device, comprising:

[0068] Memory for storing computer programs;

[0069] The processor is used to call the computer program stored in the memory and execute the document recognition method according to the obtained program.

[0070] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer-executable program for enabling a computer to execute a document recognition method. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0072] Figure 1 A schematic diagram of a system architecture provided for an embodiment of the present invention;

[0073] Figure 2 A schematic diagram of a flow chart of a document recognition method provided by an embodiment of the present invention;

[0074] Figure 3 A schematic diagram of an identity card provided by an embodiment of the present invention;

[0075] Figure 4 A schematic diagram of cutting an address text area image provided by an embodiment of the present invention;

[0076] Figure 5 A schematic diagram of another process of document recognition provided by an embodiment of the present invention;

[0077] Figure 6 A schematic diagram of the structure of a document recognition device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0079] Figure 1 A system architecture is provided for an embodiment of the present invention. Figure 1 As shown, the system architecture may be a server 100 , including a processor 110 , a communication interface 120 , and a memory 130 .

[0080] The communication interface 120 is used to communicate with the terminal device, send and receive information transmitted by the terminal device, and realize communication.

[0081] The processor 110 is the control center of the server 100, and uses various interfaces and lines to connect various parts of the entire server 100, and executes various functions of the server 100 and processes data by running or executing software programs and / or modules stored in the memory 130, and calling data stored in the memory 130. Optionally, the processor 110 may include one or more processing units.

[0082] The memory 130 can be used to store software programs and modules. The processor 110 executes various functional applications and data processing by running the software programs and modules stored in the memory 130. The memory 130 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function, etc.; the data storage area can store data created according to business processing, etc. In addition, the memory 130 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0083] It should be noted that the above Figure 1 The structure shown is only an example and is not limited to this embodiment of the present invention.

[0084] Based on the above description, Figure 2 The process of a document recognition method provided by an embodiment of the present invention is exemplarily shown, and the process can be executed by a document recognition device.

[0085] like Figure 2 As shown in the figure, the process specifically includes:

[0086] Step 201, obtaining a picture of the document to be identified.

[0087] In an embodiment of the present invention, the document to be identified may include an identity card, a vehicle registration certificate, a driver's license, a passport, a Hong Kong and Macau Pass, a Mainland Residents' Taiwan Pass, a social security card, etc. Specifically, when a user needs to handle finance, banking, examination qualification review, study abroad, or needs to handle related business on a government agency website online, he needs to submit a corresponding document image so that the staff can verify the corresponding document. For example, when user A handles banking business online, user A is required to upload a picture of his ID card. After user A uploads his ID card picture and the corresponding text is recognized by the document, the bank business reviewer reviews the identity information of user A to ensure the security of this business operation. Of course, it is not limited to this in actual application scenarios.

[0088] Step 202, performing target detection on the image of the document to be identified, determining the document area of ​​the document to be identified and the keywords in the document area, and determining the positions of the keywords.

[0089] In an embodiment of the present invention, a picture of a document to be identified is input into a multi-target detection model for identification, and the document area of ​​the document to be identified and the keywords in the document area are determined; the multi-target detection model is determined by training a preset instance segmentation model using a first document image sample set. And based on the document area of ​​the document to be identified and the keywords in the document area, the position of the keywords is determined. Specifically, when the multi-target detection model is used to perform target detection on the picture of the document to be identified, the document area of ​​the document to be identified and the keywords in the document area can be detected at the same time, that is, while detecting the document area of ​​the document to be identified, some keywords fixed on the document area are also detected as targets, and based on the keywords in the document area, the position of the keywords is determined. It should be noted that since the image of the document to be identified includes not only the document area of ​​the document to be identified itself, but also the background part when the document to be identified is photographed, it is necessary to detect the document area of ​​the document to be identified when performing target detection on the image of the document to be identified, so that the text on the document to be identified can be accurately identified when the text part of the document to be identified is subsequently identified.

[0090] In addition, when the first ID card image sample set is used to train the preset instance segmentation model to determine the multi-target detection model, the first ID card image sample set is first obtained, and then the keywords in the first ID card image sample set are annotated to obtain the annotated first ID card image sample set, and the annotated first ID card image sample set is vectorized to obtain the sample vector set corresponding to the annotated first ID card image sample set. Then, the sample vector set corresponding to the annotated first ID card image sample set is input into the preset instance segmentation model for training to obtain the multi-target detection model. Since the target features of the ID card to be identified are obvious, a relatively small number of ID card image sample sets are required to achieve a good target detection effect, that is, the ID card area of ​​the ID card to be identified and the keywords in the ID card area can be well detected at the same time. Therefore, the number of the first ID card image sample sets used is relatively small. For example, when using the ID card training sample to train the preset instance segmentation model, it is sufficient to obtain the ID cards of several internal staff members, which well solves the problem of difficulty in obtaining ID card samples and high text annotation cost in the prior art. By labeling the keywords on the ID cards of several internal staff members, and performing vector processing on the labeled ID cards of several internal staff members, a sample vector set corresponding to the labeled ID cards of several internal staff members is obtained, and then the sample vector set corresponding to the labeled ID cards of several internal staff members is input into a preset instance segmentation model for training, and a multi-target detection model for ID cards is obtained. The preset instance segmentation model may include YOLACT, Mask R-CNN, BlendMask, etc.

[0091] For example, when the document to be identified is an ID card, for example, when user B applies for studying abroad online, he needs to upload his ID card picture for verification by the reviewer. After user B uploads his ID card picture, the document recognition system uses the deep learning instance segmentation algorithm to perform target detection on the ID card picture uploaded by user B, and detects the ID card area of ​​the ID card picture uploaded by user B and the keywords of name, gender, birth, ethnicity, address, and citizen ID number in the ID card area as targets, and can obtain the corresponding positions of these keywords.

[0092] Step 203: Process the position of the keyword using a preset rule to determine the text area corresponding to the keyword.

[0093] In the embodiment of the present invention, based on the position of the keyword, the region where the keyword is located is scaled according to a preset scaling ratio to determine the text region corresponding to the keyword. Specifically, taking the ID card as an example, based on the positions of several keywords in the region of the ID card determined, the regions where the keywords are located are scaled according to the preset scaling ratios corresponding to the keywords to obtain the text regions corresponding to the keywords. For example, Figure 3 As shown in the ID card schematic diagram, the multi-target detection model is used to detect the position of the name and the text box of the name in the ID card area, and the text box of the keyword "name" is expanded to the right, upward or downward according to a certain scaling ratio to obtain the text area of ​​the entire line of "Name XXX". This can replace text detection and accurately and quickly obtain the text area corresponding to the keyword.

[0094] In addition, if there are multiple lines or columns of text in the text area corresponding to the keyword, it is necessary to determine whether there are multiple lines or columns of text in the text area corresponding to the keyword, and when it is determined that there are multiple lines or columns of text in the text area corresponding to the keyword, the text area corresponding to the keyword is screenshotted to obtain a text area image corresponding to the keyword, and then the text area image corresponding to the keyword is grayed to obtain a blank part, and the blank part is cut to obtain multiple text areas corresponding to the keyword. Specifically, after obtaining the text area image corresponding to the keyword, the text area image corresponding to the keyword is first grayed to obtain the gray value of each pixel in the text area image corresponding to the keyword. Then, the gray value of each pixel in the text area image corresponding to the keyword is binarized to obtain the gray value of each pixel in the text area image corresponding to the keyword after binarization, and the gray values ​​of all pixels in each row or column in the text area image corresponding to the keyword after binarization are averaged to obtain the projection gray value of each row or column in the text area image corresponding to the keyword after binarization. For any row in the multiple rows or any column in the multiple columns in the text area image corresponding to the keyword after binarization, the projected grayscale value of the row or column is compared with the preset threshold value. If the projected grayscale value of the row or column is greater than or equal to the preset threshold value, the row or column is determined to be a blank part, otherwise the row or column is determined to be a text part, and the blank part between rows or columns is cut to obtain multiple text areas corresponding to the keyword. The preset threshold value can be set according to experience.

[0095] For example, let's take the ID card as an example. Figure 3As shown, since the text area corresponding to the keyword "address" has multiple lines, in order to facilitate the subsequent text recognition model to accurately identify the text content corresponding to the text area, the text area corresponding to the keyword "address" with multiple lines is cut, that is, the address text area framed by the preset rules is screenshotted, and the color space conversion function cvtColor of OpenCV (Open Source Computer Vision Library) is used to grayscale the address text area image, that is, each pixel in the address text area image (each image is composed of a pixel matrix) is converted from three RGB values ​​to one value (grayscale processing is to make each pixel in the pixel matrix satisfy the relationship: R=G=B, and this equal value is the grayscale value). Then, the adaptive threshold function adaptiveThreshold is used to binarize the grayscale value of each pixel on the address text area image, that is, the grayscale value of each pixel in the pixel matrix on the address text area image is 0 (black) or 255 (white), so that the entire image presents only black and white effects. For example, when the grayscale value is greater than 160, the grayscale value is binarized and set to 255, and when the grayscale value is less than 160, the grayscale value is binarized and set to 0. Among them, the color of a pixel is represented by three RGB values ​​(R: Red, G: Green, B: Blue), so a pixel matrix corresponds to three color vector matrices, and the value range of R, G, and B is 0 to 255; the value 255 is white, 0 is black, and 0-255 represents the depth of color.

[0096] After binarizing the grayscale value of each pixel on the address text area image, average the grayscale values ​​of all pixels in each row of the address text area image to obtain the projected grayscale value of each row on the address text area image (for example, if the address text box width * height = 500 * 150, it means that each row has 500 grayscale values, and the average of these 500 grayscale values ​​can be used to obtain the projected grayscale value of each row). Then compare the projected grayscale value of each row with the preset threshold (for example, assuming the preset threshold is 240, the projected grayscale value of a row is 255, and the projected grayscale value of the row is greater than the preset threshold, then the row is considered to be a blank part; if the projected grayscale value of a row is 100, then the projected grayscale value of the row is less than the preset threshold, then the row is considered to be a text part). In this way, the blank part between rows can be detected, and the blank part can be cut from the middle to separate each text line in the address text area, and the following is obtained. Figure 4 The three lines of text shown.

[0097] It should be noted that if Figure 5As shown, the method adopted in the embodiment of the present invention combines the document detection, keyword detection and preset rules together when detecting the image of the document to be identified, replacing the document detection and text detection methods in the prior art, and can simultaneously determine the document area of ​​the document to be identified, the location of the keyword, and the text area corresponding to the keyword, and then identify the text area corresponding to the keyword to identify the text content corresponding to the text area. In this way, the document recognition process in the prior art can be simplified, the performance of document recognition can be improved, and the annotation cost of text detection can be reduced.

[0098] Step 204: Identify the text region corresponding to the keyword and determine the text content corresponding to the text region.

[0099] In the embodiment of the present invention, the text area corresponding to the keyword is input into the text recognition model for recognition, and the text content corresponding to the text area is determined. Figure 3 As shown, the entire text area of ​​"Name XXX" is input into the text recognition model for recognition, and the corresponding text content "Name XXX" can be recognized, or the entire text area of ​​"Birth XX Year XX Month XX Day" is input into the text recognition model for recognition, and the corresponding text content "Birth XX Year XX Month XX Day" can be recognized. The text recognition model is obtained by training the preset text recognition model using the second ID image sample set.

[0100] In addition, when the second ID card image sample set is used to train the preset text recognition model to determine the text recognition model, the second ID card image sample set is first obtained, and the text content in the second ID card image sample set is annotated to obtain the annotated second ID card image sample set. The annotated second ID card image sample set is then vectorized to obtain a sample vector set corresponding to the annotated second ID card image sample set. The sample vector set corresponding to the annotated second ID card image sample set is then input into the preset text recognition model for training to obtain the text recognition model.

[0101] The above embodiments show that by performing target detection on the image of the certificate to be identified, the certificate area of ​​the certificate to be identified and the keywords in the certificate area can be detected quickly and accurately at the same time, and based on the certificate area of ​​the certificate to be identified and the keywords in the certificate area, the position of the keyword can be accurately determined, and the position of the keyword is processed using a preset rule to determine the text area corresponding to the keyword. After that, the text area corresponding to the keyword is identified, and the text content corresponding to the text area can be accurately and quickly determined. Since the existing certificate recognition method is to perform certificate detection on the certificate to be identified and then perform text detection, resulting in a long process and low performance for the certificate detection to be identified, by putting target detection and preset rules together to replace the original certificate detection and text detection, the certificate area of ​​the certificate to be identified and the keywords in the certificate area can be detected quickly and accurately at the same time, and the text area corresponding to the keyword is framed using the preset rules, which can help improve the performance of certificate recognition and has strong versatility for the recognition of the same type of certificates.

[0102] Based on the same technical concept, Figure 6 An example of a document recognition device provided by an embodiment of the present invention is shown, and the device can execute the process of the document recognition method.

[0103] like Figure 6 As shown, the device comprises:

[0104] An acquisition unit 601 is used to acquire a picture of a document to be identified;

[0105] The processing unit 602 is used to perform target detection on the image of the document to be identified, determine the document area of ​​the document to be identified and the keywords in the document area, and determine the position of the keywords; use preset rules to process the position of the keywords to determine the text area corresponding to the keywords; identify the text area corresponding to the keywords, and determine the text content corresponding to the text area.

[0106] Optionally, the processing unit 602 is specifically configured to:

[0107] Inputting the image of the document to be identified into a multi-target detection model for identification, and determining the document area of ​​the document to be identified and the keywords in the document area; the multi-target detection model is determined by training a preset instance segmentation model using the first document image sample set;

[0108] Based on the document area of ​​the document to be identified and the keyword in the document area, the position of the keyword is determined.

[0109] Optionally, the processing unit 602 is specifically configured to:

[0110] Based on the position of the keyword, the area where the keyword is located is scaled according to a preset scaling ratio to determine the text area corresponding to the keyword.

[0111] Optionally, the processing unit 602 is further configured to:

[0112] After determining the text area corresponding to the keyword, determining whether there are multiple rows or columns of text in the text area corresponding to the keyword;

[0113] If so, the text area corresponding to the keyword is segmented using a preset segmentation rule to determine multiple text areas corresponding to the keyword; otherwise, a single row or single column text area corresponding to the keyword is determined as the text area corresponding to the keyword.

[0114] Optionally, the processing unit 602 is specifically configured to:

[0115] Taking a screenshot of the text area corresponding to the keyword to obtain a picture of the text area corresponding to the keyword;

[0116] Grayscale processing is performed on the text area image corresponding to the keyword to obtain a blank part, and the blank part is cut to obtain multiple text areas corresponding to the keyword.

[0117] Optionally, the processing unit 602 is specifically configured to:

[0118] Obtaining the first certificate image sample set;

[0119] Annotating keywords in the first ID image sample set to obtain an annotated first ID image sample set;

[0120] Performing vectorization processing on the labeled first ID image sample set to obtain a sample vector set corresponding to the labeled first ID image sample set;

[0121] The sample vector set corresponding to the labeled first certificate image sample set is input into the preset instance segmentation model for training to obtain the multi-target detection model.

[0122] Optionally, the processing unit 602 is further configured to:

[0123] Before identifying the text area corresponding to the keyword, obtaining a second document image sample set;

[0124] Annotating text content in the second ID image sample set to obtain an annotated second ID image sample set;

[0125] Performing vectorization processing on the labeled second ID image sample set to obtain a sample vector set corresponding to the labeled second ID image sample set;

[0126] Inputting the sample vector set corresponding to the annotated second certificate image sample set into a preset text recognition model for training to obtain a text recognition model;

[0127] The processing unit 602 is specifically used for:

[0128] The text area corresponding to the keyword is input into the text recognition model for recognition, and the text content corresponding to the text area is determined.

[0129] Based on the same technical concept, an embodiment of the present invention provides a computing device, including:

[0130] Memory for storing computer programs;

[0131] The processor is used to call the computer program stored in the memory and execute the document recognition method according to the obtained program.

[0132] Based on the same technical concept, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer-executable program, and the computer-executable program is used to enable a computer to execute a document recognition method.

[0133] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0135] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0136] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0137] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0138] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of this application and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A document recognition method, characterized in that: include: Get the image of the document to be identified; Performing target detection on the image of the document to be identified, determining the document area of ​​the document to be identified and fixed keywords in the document area, and determining the position of the keywords; Based on the position of the keyword, the area where the keyword is located is framed into a text box according to a preset zoom ratio, and the text area corresponding to the keyword is determined, and the area in the text box is the text area; Identifying a text area corresponding to the keyword and determining text content corresponding to the text area; The performing target detection on the image of the document to be identified, determining the document area of ​​the document to be identified and the keywords in the document area, and determining the positions of the keywords, includes: Inputting the image of the document to be identified into a multi-target detection model for identification, and determining the document area of ​​the document to be identified and the keywords in the document area; the multi-target detection model is determined by training a preset instance segmentation model using the first document image sample set; Based on the document area of ​​the document to be identified and the keyword in the document area, the position of the keyword is determined.

2. The method according to claim 1, characterized in that After determining the text area corresponding to the keyword, the method further includes: Determine whether there are multiple lines or columns of text in the text area corresponding to the keyword; If so, the text area corresponding to the keyword is segmented using a preset segmentation rule to determine multiple text areas corresponding to the keyword; otherwise, a single row or single column text area corresponding to the keyword is determined as the text area corresponding to the keyword.

3. The method according to claim 2, characterized in that The using of a preset cutting rule to cut the text area corresponding to the keyword to determine a plurality of text areas corresponding to the keyword includes: Taking a screenshot of the text area corresponding to the keyword to obtain a picture of the text area corresponding to the keyword; Grayscale the text area image corresponding to the keyword to obtain the grayscale value of each pixel in the text area; Determine the projection gray value of each line in the text area according to the gray value of each pixel in the text area; If the projection gray value of any line in the text area is greater than a preset threshold, the line is regarded as a blank part, and the blank part is cut to obtain a plurality of text areas corresponding to the keywords.

4. The method according to claim 1, characterized in that The step of using the first document image sample set to train a preset instance segmentation model to determine the multi-target detection model includes: Obtaining the first certificate image sample set; Annotating keywords in the first ID image sample set to obtain an annotated first ID image sample set; Performing vectorization processing on the labeled first ID image sample set to obtain a sample vector set corresponding to the labeled first ID image sample set; The sample vector set corresponding to the labeled first certificate image sample set is input into the preset instance segmentation model for training to obtain the multi-target detection model.

5. The method according to any one of claims 1 to 4, characterized in that: Before identifying the text area corresponding to the keyword, the method further includes: Obtain a second document image sample set; Annotating text content in the second ID image sample set to obtain an annotated second ID image sample set; Performing vectorization processing on the labeled second ID image sample set to obtain a sample vector set corresponding to the labeled second ID image sample set; Inputting the sample vector set corresponding to the annotated second certificate image sample set into a preset text recognition model for training to obtain a text recognition model; The identifying the text area corresponding to the keyword and determining the text content corresponding to the text area includes: The text area corresponding to the keyword is input into the text recognition model for recognition, and the text content corresponding to the text area is determined.

6. A document recognition device, characterized in that: include: An acquisition unit, used to acquire a picture of a document to be identified; The processing unit is used to perform target detection on the image of the document to be identified, determine the document area of ​​the document to be identified and the fixed keywords in the document area, and determine the position of the keyword; based on the position of the keyword, according to a preset zoom ratio, frame the area where the keyword is located into a text box, determine the text area corresponding to the keyword, and the area in the text box is the text area; identify the text area corresponding to the keyword, and determine the text content corresponding to the text area; The processing unit is further used to input the image of the document to be identified into the multi-target detection model for identification, and determine the document area of ​​the document to be identified and the keywords in the document area; The multi-target detection model is determined by training a preset instance segmentation model using a first document image sample set; based on the document area of ​​the document to be identified and the keywords in the document area, the position of the keyword is determined.

7. A computing device, characterized in that include: Memory for storing computer programs; A processor, configured to call a computer program stored in the memory, and execute the method according to any one of claims 1 to 5 according to the obtained program.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer-executable program, and the computer-executable program is used to enable a computer to execute the method according to any one of claims 1 to 5.

Images