Handwritten document processing method and device, equipment, medium and program product

Abstract

The invention discloses a handwritten document processing method, device and equipment, a medium and a program product. The method comprises the following steps: acquiring a handwritten document image, and carrying out standardization processing on the handwritten document image to obtain a standard handwritten document; determining a target document template, a target information system and an application page according to the standard handwritten document; according to the target document template, identifying the handwritten content of each information item in the standard handwritten document, and obtaining an identification result of each information item; and according to the target information system, checking the identification result of the first information item, and after the identification result of the first information item is checked to be correct, performing data filling on the application page according to the identification result of each information item. By the adoption of the technical scheme, the handwritten content in the handwritten document can be accurately recognized, the recognition efficiency is improved, automatic checking and filling of information can be achieved, and the business handling efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of financial technology, and in particular to a method, device, equipment, medium and program product for processing handwritten documents. Background Art

[0002] In the process of handling banking, reimbursement, visa and other businesses, users are often required to fill out documents manually, which are then manually reviewed and entered into the system by dedicated business processing personnel. The business processing process is time-consuming and easily affected by human factors. When the user's handwriting is unclear or filled in incorrectly, it may lead to information entry errors, thereby reducing business processing efficiency and even causing economic losses or personnel disputes.

[0003] In the prior art, optical character recognition (OCR) technology is generally used to directly extract and recognize the features of each handwritten content in handwritten documents. The recognition results of the existing recognition methods are not sequential. Although most text information can be extracted, the extracted information is disorganized and has poor readability. In addition, there is also the problem of incomplete extraction of some content. Summary of the Invention

[0004] The present invention provides a method, device, equipment, medium and program product for processing handwritten documents, which can accurately recognize the handwritten content in handwritten documents, improve recognition efficiency, and realize automatic verification and filling of information, thereby improving business processing efficiency.

[0005] According to one aspect of the present invention, a method for processing handwritten documents is provided, comprising:

[0006] Collect handwritten document images and perform standardization processing on the handwritten document images to obtain standard handwritten documents;

[0007] Determine the target document template, target information system, and application page based on the standard handwritten document;

[0008] According to the target document template, the handwritten content of each information item in the standard handwritten document is recognized to obtain the recognition result of each information item;

[0009] The recognition result of the first information item is checked according to the target information system, and after the recognition result of the first information item is checked to be correct, the application page is filled with data according to the recognition results of each information item.

[0010] Optionally, perform standardization processing on the handwritten document image to obtain a standard handwritten document, including:

[0011] Use the median filter algorithm to update the pixel value of each pixel in the handwritten document image;

[0012] Binarize each pixel according to the pre-selected global threshold and the updated pixel value of each pixel;

[0013] Corner points and edge feature points of the handwritten document image are extracted, and tilt correction is performed on the handwritten document image based on the corner points and edge feature points.

[0014] The advantage of this setting is that by using the median filtering algorithm to denoise the handwritten document image, compared with other denoising methods such as mean filtering, it can better maintain the edge information of the image while removing noise. The median filtering algorithm is more flexible in processing edge pixels, especially for bank counter business processing scenarios. The median filtering algorithm can adapt to the noise caused by factors such as lighting and document flatness, and can freely adjust the size and shape of the filter window. It has low computational complexity and high efficiency, which is very consistent with the business needs of bank counters. By performing tilt correction on the handwritten document image, a regular handwritten document can be obtained, the information can be completely retained and highlighted, and it can help improve the document recognition rate.

[0015] Optionally, based on the standard handwritten document, determining a target document template, a target information system, and an application page includes:

[0016] Using OCR technology to recognize standard handwritten documents, obtaining a first recognition result, and determining a target document template based on the first recognition result and a pre-established document identification library;

[0017] Determine the target information system and application page based on the target document template.

[0018] The advantage of this setting is that it can directly determine the information system and application page based on the standardized handwritten document image, thereby improving the efficiency of information verification and filling, effectively reducing the manual operation items of business personnel, and improving business processing efficiency.

[0019] Optionally, the target document template includes the target document base, user handwriting areas and fixed information areas pre-divided in the target document base, and information items corresponding to the user handwriting areas;

[0020] Based on the target document template, the handwritten content of each information item in the standard handwritten document is recognized, and the recognition results of each information item are obtained, including:

[0021] Determining, in a standard handwritten document according to the target document template, a plurality of user handwriting areas and at least one fixed information area corresponding to each user handwriting area;

[0022] Segment the standard handwritten document based on the user's handwritten area, obtain multiple segmented image modules, and perform any of the following:

[0023] When the user handwriting area belongs to the information selection type, the association between the handwriting content and each option is identified in the image module, and the recognition result of the information item corresponding to the user handwriting area is obtained based on the association between the handwriting content and each option;

[0024] When the user handwriting area belongs to the information filling type, the fixed information area is removed in the image module to obtain the handwritten content in the user handwriting area, and the handwritten content in the user handwriting area is subjected to OCR recognition. According to the OCR recognition result, the recognition result of the information item corresponding to the user handwriting area is obtained.

[0025] The advantage of this setting is that by dividing the handwritten information area and the fixed information area, it is possible to effectively retain the handwritten content while removing the printed content as much as possible, thereby improving the accuracy of OCR recognition. Moreover, by dividing the information selection type and the information filling type, it is possible to improve the recognition accuracy of the selected information, and only recognize the necessary handwritten content, further improving the recognition accuracy and efficiency.

[0026] Optionally, removing the fixed information area in the image module to obtain the handwritten content in the user handwriting area includes:

[0027] Identify whether the handwritten content overlaps with the fixed information area;

[0028] If so, identifying feature points of the fixed information area in the image module, performing pixel conversion on the feature points of the fixed information area, completing the handwritten content in the image module after the pixel conversion, and determining the completed handwritten content as the handwritten content in the user handwriting area;

[0029] If not, the fixed information area is cropped in the image module, and the remaining content in the image module is determined as the handwritten content in the user handwriting area.

[0030] The advantage of this setting is that: considering that some users will overlap their handwritten content with the fixed information area when filling out forms, in order to avoid inaccurate recognition results in this case, two different processing solutions are designed for overlapping and non-overlapping situations. This can improve recognition efficiency and accuracy, and provide users and business personnel with a good business processing experience.

[0031] Optionally, after verifying the recognition result of the first information item according to the target information system, the method further includes:

[0032] When it is determined that the recognition result of the target first information item does not match the record value of the target first information item in the target information system, calculating a similarity between the record value of the target first information item and the recognition result;

[0033] When the similarity is greater than a preset similarity threshold, generating correction prompt information of the target first information item according to the recorded value;

[0034] Otherwise, an error prompt of the target first information is directly generated.

[0035] The advantage of this setting is that for some errors caused by handwritten errors, friendly prompts can be provided to users to ensure the accuracy of business information. For errors not caused by handwritten errors, information security can be effectively guaranteed, error prompts can be directly issued, business processing can be suspended, and information leakage or property loss can be avoided.

[0036] According to another aspect of the present invention, there is provided a handwritten document processing device, comprising:

[0037] The handwritten document collection and processing module is used to collect handwritten document images and perform standard processing on the handwritten document images to obtain standard handwritten documents;

[0038] A system information determination module, configured to determine a target document template, a target information system, and an application page based on the standard handwritten document;

[0039] A handwritten content recognition module is used to recognize the handwritten content of each information item in a standard handwritten document according to a target document template and obtain recognition results for each information item;

[0040] The data checking and filling module is used to check the recognition result of the first information item according to the target information system, and after the recognition result of the first information item is checked to be correct, fill in the data on the application page according to the recognition results of each information item.

[0041] According to another aspect of the present invention, an electronic device is provided, comprising:

[0042] at least one processor; and

[0043] a memory communicatively connected to the at least one processor; wherein,

[0044] The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can execute the handwritten document processing method according to any embodiment of the present invention.

[0045] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the handwritten document processing method according to any embodiment of the present invention when executed.

[0046] According to another aspect of the present invention, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the handwritten document processing method according to any embodiment of the present invention is implemented.

[0047] The technical solution of the embodiment of the present invention, by collecting handwritten document images and performing standard processing on the handwritten document images to obtain standard handwritten documents, can better maintain the edge information of the image while removing noise, can obtain regular handwritten documents, and the information can be completely retained and highlighted, and can help improve the recognition rate of documents. By determining the target document template, target information system and application page according to the standard handwritten document, according to the target document template, the handwritten content of each information item in the standard handwritten document is recognized, and the recognition result of each information item is obtained. According to the target information system, the recognition result of the first information item is verified, and after the recognition result of the first information item is verified to be correct, the application page is filled with data according to the recognition results of each information item. This method can directly determine the information system and application page according to the standardized handwritten document image, and can accurately recognize the handwritten content, thereby realizing automatic verification and automatic information filling, improving the efficiency of information verification and filling, effectively reducing the manual operation items of business personnel, and improving business processing efficiency.

[0048] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0050] Figure 1 This is a flow chart of a method for processing handwritten documents provided according to the first embodiment of the present invention;

[0051] Figure 2 is a schematic diagram of a handwritten document image provided according to an embodiment of the present invention;

[0052] Figure 3 is a schematic diagram of a standard handwritten document provided according to an embodiment of the present invention;

[0053] Figure 4 is a schematic diagram of a target document template provided according to an embodiment of the present invention;

[0054] Figure 5 is a schematic diagram of segmenting a user handwriting area in an image module according to an embodiment of the present invention;

[0055] Figure 6 is a flowchart of another method for processing handwritten documents provided according to the second embodiment of the present invention;

[0056] Figure 7 This is a schematic structural diagram of a handwritten document processing device provided according to a third embodiment of the present invention;

[0057] Figure 8 The present invention is a schematic diagram of the structure of an electronic device for implementing the handwritten document processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0059] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0060] Example 1

[0061] Figure 1This is a flow chart of a method for processing handwritten documents provided in the first embodiment of the present invention. This embodiment is applicable to the case where handwritten content in a handwritten document is recognized and the recognized handwritten content is matched with an information item. The method can be executed by a handwritten document processing device, which can be implemented in the form of hardware and / or software and can generally be configured in a computer or processor with data processing capabilities. Figure 1 As shown, the method includes:

[0062] S110 , collecting a handwritten document image, and performing standardization processing on the handwritten document image to obtain a standard handwritten document.

[0063] Optionally, a handwritten document may refer to a paper voucher filled out manually in a certain format and purpose, used to record various economic transactions or transaction information, usually containing key information such as date, amount, project name, person in charge, etc., such as reimbursement form, receipt, delivery note, bank transaction form, etc.

[0064] Optionally, in the embodiment described in the present invention, a detailed explanation can be given by taking the remittance slip in the bank business form as an example, and the present invention is highly adaptable to the banking business scenario, effectively solving the problem of low business processing efficiency caused by the need for business personnel to perform manual information verification and system entry in the existing banking business scenario, and can also effectively avoid errors in the process of information verification or entry.

[0065] Optionally, the handwritten document processing device can be in various forms such as a plug-in, a web page, or an independent system. If it is in the form of a plug-in, it can be configured in the business system that manages the application content of handwritten documents. For example, when the handwritten document is a bank remittance form, the handwritten document processing device can be configured in the banking business system in the form of a plug-in.

[0066] Optionally, images of handwritten documents can be captured and uploaded using a high-resolution camera or scanner. When the handwritten document is a bank business document, the high-resolution camera can be a high-resolution scanner commonly used in banking business scenarios, without the need to upgrade or modify existing shooting equipment.

[0067] Optionally, the standardization process may include denoising, binarization, and tilt correction. The standard handwritten document may refer to a document image obtained after the standardization process is performed on the handwritten document image.

[0068] The standardization of the handwritten document image to obtain a standard handwritten document may include:

[0069] Use the median filter algorithm to update the pixel value of each pixel in the handwritten document image;

[0070] Binarize each pixel according to the pre-selected global threshold and the updated pixel value of each pixel;

[0071] The corner points and edge feature points of the handwritten document image are extracted, and the handwritten document image is tilted corrected based on the corner points and edge feature points.

[0072] Optionally, since the shooting process may be affected by various factors such as lighting, equipment, and paper quality, the handwritten document image may contain interference information such as noise, spots, and lines. Therefore, the handwritten document image needs to be denoised to reduce the impact of noise on the handwritten document image and improve the accuracy of subsequent handwritten content recognition.

[0073] Optionally, using a median filtering algorithm, the value of each pixel in the handwritten document image can be replaced with the median value of all pixel values ​​in its neighborhood. The neighborhood of the pixel can be determined based on a pre-selected window size. The window size is an odd number, and the number of pixels in the window in the vertical and horizontal directions is generally selected as a smaller value, such as 3X3, 5X5. The odd window size can facilitate the confirmation of the middle value. The smaller the window value, the more detailed information in the handwritten document image can be retained, thereby improving the accuracy of handwritten content recognition.

[0074] Optionally, the pixel value of each pixel point in the handwritten document image is updated using a median filtering algorithm, which may include: sequentially obtaining all pixel values ​​within the window of each pixel in the handwritten document image; using a quick sorting algorithm to sort all pixel values ​​within the window of the current pixel in ascending order; selecting the middle value from the sorted pixel values ​​as the filtered value of the current pixel, and using the filtered value to update the pixel value of the current pixel until the pixel value update of all pixels in the handwritten document image is completed.

[0075] Alternatively, the quick sort algorithm has lower time complexity and can significantly improve sorting efficiency.

[0076] Optionally, binarization of each pixel point according to a pre-selected global threshold and the updated pixel value of each pixel point may include:

[0077] Determine whether the updated pixel value of each pixel is greater than the global threshold; if so, change the pixel value of the pixel to the first pixel value; otherwise, change the pixel value of the pixel to the second pixel value.

[0078] Optionally, the first pixel value can be 255 and the second pixel value can be 0. Then, pixels greater than the global threshold are set to white, and pixels less than the global threshold are set to black. This allows for simple and quick conversion into a black and white image, which is particularly suitable for documents with a white background, such as bank remittance slips.

[0079] It is understandable that due to reasons such as shooting angle or device shaking, the handwritten document image may appear tilted. In order to ensure that the text and information in the image are in the correct position and facilitate subsequent OCR recognition and information extraction, the handwritten document in the handwritten document image needs to be tilted.

[0080] Optionally, corner points are feature points at the four corners of a handwritten document in a handwritten document image, and edge feature points are feature points that can be extracted from each edge of a handwritten document. Based on the corner points and edge feature points, the handwritten document can be located in the handwritten document image. The corner points and edge feature points can be realized through the automatic edge detection function in the business system. The business system is a system for processing handwritten document declaration business.

[0081] Optionally, performing tilt correction on the handwritten document image based on the corner points and edge feature points may include: calculating the four desired vertex coordinates of the handwritten document after tilt correction based on the four corner points, and determining the target rectangle based on the four desired vertex coordinates after tilt correction; calculating the perspective transformation matrix from the handwritten document image to the target rectangle based on the handwritten document and the target rectangle in the handwritten document; and performing perspective transformation on the handwritten document based on the perspective transformation matrix to perform tilt correction on the handwritten document image.

[0082] Figure 2 is a schematic diagram of an optional handwritten document image. Figure 3 This is a schematic diagram of a standard handwritten document. Figure 2 、 Figure 3 As shown, the handwritten document image is a bank remittance slip. Figure 2 The bank remittance slips taken by the high-definition camera are skewed and have uneven colors. Figure 3 This is a bank remittance slip that has been standardized. The standardized bank remittance slip is converted into a black and white image with a regular shape and uniform color distribution.

[0083] The advantage of this setting is that by using the median filtering algorithm to denoise the handwritten document image, compared with other denoising methods such as mean filtering, it can better maintain the edge information of the image while removing noise. The median filtering algorithm is more flexible in processing edge pixels, especially for bank counter business processing scenarios. The median filtering algorithm can adapt to the noise caused by factors such as lighting and document flatness, and can freely adjust the size and shape of the filter window. It has low computational complexity and high efficiency, which is very consistent with the business needs of bank counters. By performing tilt correction on the handwritten document image, a regular handwritten document can be obtained, the information can be completely retained and highlighted, and it can help improve the document recognition rate.

[0084] S120. Determine the target document template, target information system, and application page based on the standard handwritten document.

[0085] Among them, based on the standard handwritten document, determining the target document template, target information system and application page may include:

[0086] Using OCR technology to recognize standard handwritten documents, obtaining a first recognition result, and determining a target document template based on the first recognition result and a pre-established document identification library;

[0087] Determine the target information system and application page based on the target document template.

[0088] Optionally, the target document template may refer to an electronic document without any content filled in, the target information system may refer to a business system that can query the information in the standard handwritten document, and the application page may refer to a page that needs to fill in information based on the standard handwritten document. For example, the target document template may be an electronic version of a bank remittance form, the target information system may be a bank business system, and the application page may refer to the remittance information filling page in the business system.

[0089] Optionally, OCR technology is a technology that converts text in an image into editable text. Using OCR technology, printed content in a standard handwritten document can be recognized, and the recognition result of the printed content can be the first recognition result.

[0090] Optionally, the document identification library may include identification information that matches each document template respectively. The identification information may include a document identification, key data items, and the like.

[0091] Optionally, after obtaining the first recognition result of the standard handwritten document, the first recognition result can be matched with the identification information in the document identification library to determine the target document template. For example, in the document identification library, the identification information that matches the remittance slip template may include the payee's account name, the payee's account number and the name of the beneficiary bank. When the first recognition result contains the above information at the same time, the target document template is determined to be the remittance slip template.

[0092] Optionally, the document template may have a corresponding relationship with the information system and the application page. After determining the target document template, the target information system and application page may be directly determined.

[0093] The advantage of this setting is that it can directly determine the information system and application page based on the standardized handwritten document image, thereby improving the efficiency of information verification and filling, effectively reducing the manual operation items of business personnel, and improving business processing efficiency.

[0094] S130 . Recognize the handwritten content of each information item in the standard handwritten document according to the target document template, and obtain the recognition result of each information item.

[0095] Optionally, an information item may refer to a basic unit in a standard handwritten document used to describe and distinguish different information contents. For example, in a bank remittance form, an information item may include the payee's account name, payee's account number, beneficiary bank name, currency, payee's account number, amount, signature and other information, which are not listed here one by one.

[0096] It is understandable that if existing handwriting recognition technology is used, it is generally impossible to fully recognize the handwriting. Moreover, since a line of a document may include multiple information items at the same time, the recognized handwriting may not correspond to the information item.

[0097] The target document template includes the target document base, user handwriting areas and fixed information areas pre-divided in the target document base, and information items corresponding to the user handwriting areas.

[0098] Optional, Figure 4 This is a schematic diagram of a target document template. Figure 4 As shown, the rectangle represented by the solid line can be a pre-divided user handwriting area, and the rectangle represented by the dotted line can be a pre-divided fixed information area. Figure 4 Only some areas in the template are divided, aiming to illustrate the user handwriting area and the fixed information area, without making specific restrictions on the divided areas.

[0099] Optionally, each user handwriting area corresponds to at least one information item.

[0100] Recognizing the handwritten content of each information item in the standard handwritten document according to the target document template and obtaining the recognition result of each information item may include:

[0101] Determining, in a standard handwritten document according to a target document template, a plurality of user handwriting areas and at least one fixed information area corresponding to each user handwriting area;

[0102] Segment the standard handwritten document based on the user's handwritten area, obtain multiple segmented image modules, and perform any of the following:

[0103] When the user handwriting area belongs to the information selection type, the association between the handwriting content and each option is identified in the image module, and the recognition result of the information item corresponding to the user handwriting area is obtained based on the association between the handwriting content and each option;

[0104] When the user handwriting area belongs to the information filling type, the fixed information area is removed in the image module to obtain the handwritten content in the user handwriting area, and the handwritten content in the user handwriting area is subjected to OCR recognition. According to the OCR recognition result, the recognition result of the information item corresponding to the user handwriting area is obtained.

[0105] Optionally, the coordinate mapping relationship between the standard handwritten document and the target document template can be obtained, and then based on the coordinate mapping relationship and the user handwriting area and fixed information area pre-divided in the target document template, multiple user handwriting areas and at least one fixed information area corresponding to each user handwriting area are determined in the standard handwritten document, and each image module is the user handwriting area obtained after segmentation.

[0106] Optionally, the user handwriting area can be pre-divided into information filling type or information selection type. For example, in a bank remittance form, the deposit type can be selected from multiple types such as current account, fixed-term passbook, lump-sum deposit, education savings, etc. Therefore, the user handwriting area corresponding to the deposit type is the information selection type, and the user handwriting area that requires actual filling in of numbers or text such as the payee's account name and payee account number is the information filling type.

[0107] Optionally, when the user handwriting area belongs to the information selection type, the minimum distance between the handwriting content and each option is identified in the image module, and the correlation between the handwriting content and each option is determined based on the minimum distance between the handwriting content and each option. The option with the highest correlation is determined as the target selection item, and the recognition result of the information item corresponding to the user handwriting area is the target selection item selected by the user.

[0108] Optionally, in the target document template, each user handwriting area and the fixed information area has a fixed positional relationship. After acquiring the image module, the fixed information area can be removed in the image module according to the fixed positional relationship, and the remaining content can be subjected to OCR recognition to obtain the handwritten content of the handwriting area; if the user handwriting area includes multiple information items, the handwritten content corresponding to each information item is determined based on the segmentation of the handwriting content after recognition and the pre-set order of information items in the user handwriting area.

[0109] Figure 5 It is a schematic diagram of segmenting the user's handwriting area in the image module. Figure 5 As shown, the left side is the segmented image module, and the right side is the image module in which the fixed information area is removed and the user handwriting area is retained.

[0110] The advantage of this setting is that by dividing the handwritten information area and the fixed information area, it is possible to effectively retain the handwritten content while removing the printed content as much as possible, thereby improving the accuracy of OCR recognition. Moreover, by dividing the information selection type and the information filling type, it is possible to improve the recognition accuracy of the selected information, and only recognize the necessary handwritten content, further improving the recognition accuracy and efficiency.

[0111] S140. Verify the recognition result of the first information item according to the target information system, and after the recognition result of the first information item is verified to be correct, fill in data on the application page according to the recognition results of each information item.

[0112] Optionally, the first information item is an information item pre-selected in the target document template. The first information item may be an information item whose filled-in content needs to be verified. For example, for a bank remittance form, the payee's account name, payee's account number, beneficiary bank name, etc. need to be proofread, and the information of the first information item needs to be stored in the target information system.

[0113] Optionally, after obtaining the recognition results of each information item, the handwritten content of each information item can be obtained, the standard information of each first information item can be determined in the target information system, and the handwritten content of the first information item can be compared with the standard information. When the handwritten content is exactly the same as the standard information, it is determined that the verification is correct.

[0114] Optionally, after the recognition result of the first information item is verified to be correct, each information item is filled in on the application page according to the handwritten content of each information item.

[0115] The technical solution of the embodiment of the present invention, by collecting handwritten document images and performing standard processing on the handwritten document images to obtain standard handwritten documents, can better maintain the edge information of the image while removing noise, can obtain regular handwritten documents, and the information can be completely retained and highlighted, and can help improve the recognition rate of documents. By determining the target document template, target information system and application page according to the standard handwritten document, according to the target document template, the handwritten content of each information item in the standard handwritten document is recognized, and the recognition result of each information item is obtained. According to the target information system, the recognition result of the first information item is verified, and after the recognition result of the first information item is verified to be correct, the application page is filled with data according to the recognition results of each information item. This method can directly determine the information system and application page according to the standardized handwritten document image, and can accurately recognize the handwritten content, thereby realizing automatic verification and automatic information filling, improving the efficiency of information verification and filling, effectively reducing the manual operation items of business personnel, and improving business processing efficiency.

[0116] Example 2

[0117] Figure 6 This is a flowchart of a method for processing handwritten documents provided in the second embodiment of the present invention. This embodiment specifically describes the method for processing handwritten documents based on the above embodiment. Figure 6 As shown, the method includes:

[0118] S210: Collect handwritten document images.

[0119] S220: Use a median filter algorithm to update the pixel value of each pixel in the handwritten document image.

[0120] S230 , performing binarization processing on each pixel point according to a pre-selected global threshold and the updated pixel value of each pixel point.

[0121] S240 , extracting corner points and edge feature points of the handwritten document image, and performing tilt correction on the handwritten document image according to the corner points and edge feature points.

[0122] S250: Use OCR technology to recognize the standard handwritten document, obtain a first recognition result, and determine a target document template based on the first recognition result and a pre-established document identification library.

[0123] S260. Determine the target information system and application page based on the target document template.

[0124] S270: Determine, in a standard handwritten document, a plurality of user handwriting areas and at least one fixed information area corresponding to each user handwriting area according to the target document template.

[0125] S280 . Segment the standard handwritten document according to the user handwriting area to obtain multiple segmented image modules; execute step S290 or step S2100 .

[0126] S290: When the user handwriting area belongs to the information selection type, the association between the handwriting content and each option is identified in the image module, and the recognition result of the information item corresponding to the user handwriting area is obtained based on the association between the handwriting content and each option.

[0127] S2100. When the user handwriting area belongs to the information filling type, remove the fixed information area in the image module to obtain the handwritten content in the user handwriting area, and perform OCR recognition on the handwritten content in the user handwriting area. According to the OCR recognition result, obtain the recognition result of the information item corresponding to the user handwriting area.

[0128] Removing the fixed information area in the image module to obtain the handwritten content in the user handwriting area may include:

[0129] Identify whether the handwritten content overlaps with the fixed information area;

[0130] If so, identifying feature points of the fixed information area in the image module, performing pixel conversion on the feature points of the fixed information area, completing the handwritten content in the image module after the pixel conversion, and determining the completed handwritten content as the handwritten content in the user handwriting area;

[0131] If not, the fixed information area is cropped in the image module, and the remaining content in the image module is determined as the handwritten content in the user handwriting area.

[0132] Optionally, the fixed information area is determined by the position coordinates of the pixel points and has characteristics such as length and width. The characteristic points in the fixed information area can be determined according to the coordinate positions of the printed content in the fixed information area.

[0133] Optionally, pixel conversion is performed on the feature points of the fixed information area, and the handwritten content in the image module after pixel conversion is completed, which may include: converting the pixel value of the feature point in the fixed information area into a first pixel value; identifying a second pixel value in the fixed information area in the image module, and when it is determined that the distance between the two target feature points is less than a preset distance, changing the pixel area between the two target feature points to the second pixel value; the first pixel value can be 255, the second pixel value can be 0, and the target feature point is a feature point in the fixed information area whose pixel value is the second pixel value.

[0134] The advantage of this setting is that: considering that some users will overlap their handwritten content with the fixed information area when filling out forms, in order to avoid inaccurate recognition results in this case, two different processing solutions are designed for overlapping and non-overlapping situations. This can improve recognition efficiency and accuracy, and provide users and business personnel with a good business processing experience.

[0135] S2110. Verify the recognition result of the first information item according to the target information system, and after the recognition result of the first information item is verified to be correct, fill in the data on the application page according to the recognition results of each information item.

[0136] After verifying the recognition result of the first information item according to the target information system, the method may further include:

[0137] When it is determined that the recognition result of the target first information item does not match the record value of the target first information item in the target information system, calculating a similarity between the record value of the target first information item and the recognition result;

[0138] When the similarity is greater than a preset similarity threshold, generating correction prompt information of the target first information item according to the recorded value;

[0139] Otherwise, an error prompt of the target first information is directly generated.

[0140] It can be understood that the record value of the target first information item and the shape features of the recognition result can be extracted separately. The shape features may include information such as the outline and strokes of the text, and the similarity is calculated based on the shape features. When the similarity is greater than the preset similarity threshold, it may be a user error, and a correction prompt message will be issued for the user to modify it in time. Otherwise, it means that the recognition error is caused by non-error reasons. At this time, in order to ensure information security, no correction prompt will be given and an error prompt will be returned directly.

[0141] The advantage of this setting is that for some errors caused by handwritten errors, friendly prompts can be provided to users to ensure the accuracy of business information. For errors not caused by handwritten errors, information security can be effectively guaranteed, error prompts can be directly issued, business processing can be suspended, and information leakage or property loss can be avoided.

[0142] The technical solution of the embodiment of the present invention, by collecting handwritten document images and performing standard processing on the handwritten document images to obtain standard handwritten documents, can better maintain the edge information of the image while removing noise, can obtain regular handwritten documents, and the information can be completely retained and highlighted, and can help improve the recognition rate of documents. By determining the target document template, target information system and application page according to the standard handwritten document, according to the target document template, the handwritten content of each information item in the standard handwritten document is recognized, and the recognition result of each information item is obtained. According to the target information system, the recognition result of the first information item is verified, and after the recognition result of the first information item is verified to be correct, the application page is filled with data according to the recognition results of each information item. This method can directly determine the information system and application page according to the standardized handwritten document image, and can accurately recognize the handwritten content, thereby realizing automatic verification and automatic information filling, improving the efficiency of information verification and filling, effectively reducing the manual operation items of business personnel, and improving business processing efficiency.

[0143] Example 3

[0144] Figure 7 This is a schematic diagram of the structure of a handwritten document processing device provided in the third embodiment of the present invention. Figure 7 As shown, the device includes: a handwritten document collection and processing module 310, a system information determination module 320, a handwritten content recognition module 330 and a data verification and filling module 340.

[0145] The handwritten document collection and processing module 310 is used to collect handwritten document images and perform standardization processing on the handwritten document images to obtain standard handwritten documents.

[0146] The system information determination module 320 is used to determine the target document template, target information system and application page according to the standard handwritten document.

[0147] The handwritten content recognition module 330 is used to recognize the handwritten content of each information item in the standard handwritten document according to the target document template and obtain the recognition result of each information item.

[0148] The data checking and filling module 340 is used to check the recognition result of the first information item according to the target information system, and after the recognition result of the first information item is checked to be correct, fill in the data on the application page according to the recognition results of each information item.

[0149] The technical solution of the embodiment of the present invention, by collecting handwritten document images and performing standard processing on the handwritten document images to obtain standard handwritten documents, can better maintain the edge information of the image while removing noise, can obtain regular handwritten documents, and the information can be completely retained and highlighted, and can help improve the recognition rate of documents. By determining the target document template, target information system and application page according to the standard handwritten document, according to the target document template, the handwritten content of each information item in the standard handwritten document is recognized, and the recognition result of each information item is obtained. According to the target information system, the recognition result of the first information item is verified, and after the recognition result of the first information item is verified to be correct, the application page is filled with data according to the recognition results of each information item. This method can directly determine the information system and application page according to the standardized handwritten document image, and can accurately recognize the handwritten content, thereby realizing automatic verification and automatic information filling, improving the efficiency of information verification and filling, effectively reducing the manual operation items of business personnel, and improving business processing efficiency.

[0150] Based on the above embodiments, the handwritten document collection and processing module 310 can be specifically used to:

[0151] Use the median filter algorithm to update the pixel value of each pixel in the handwritten document image;

[0152] Binarize each pixel according to the pre-selected global threshold and the updated pixel value of each pixel;

[0153] Corner points and edge feature points of the handwritten document image are extracted, and tilt correction is performed on the handwritten document image based on the corner points and edge feature points.

[0154] Based on the above embodiments, the system information determination module 320 may be specifically configured to:

[0155] Using OCR technology to recognize standard handwritten documents, obtaining a first recognition result, and determining a target document template based on the first recognition result and a pre-established document identification library;

[0156] Determine the target information system and application page based on the target document template.

[0157] Based on the above embodiments, the target document template includes the target document base, the user handwriting areas and fixed information areas pre-divided in the target document base, and the information items corresponding to the user handwriting areas;

[0158] The handwritten content recognition module 330 may include:

[0159] an area division unit, configured to determine, in a standard handwritten document according to the target document template, a plurality of user handwritten areas and at least one fixed information area corresponding to each user handwritten area;

[0160] The image module processing unit is used to segment the standard handwritten document according to the user's handwritten area, obtain multiple segmented image modules, and perform any of the following:

[0161] When the user handwriting area belongs to the information selection type, the association between the handwriting content and each option is identified in the image module, and the recognition result of the information item corresponding to the user handwriting area is obtained based on the association between the handwriting content and each option;

[0162] When the user handwriting area belongs to the information filling type, the fixed information area is removed in the image module to obtain the handwritten content in the user handwriting area, and the handwritten content in the user handwriting area is subjected to OCR recognition. According to the OCR recognition result, the recognition result of the information item corresponding to the user handwriting area is obtained.

[0163] Based on the above embodiments, the image module processing unit can be specifically used to:

[0164] Identify whether the handwritten content overlaps with the fixed information area;

[0165] If so, identifying feature points of the fixed information area in the image module, performing pixel conversion on the feature points of the fixed information area, completing the handwritten content in the image module after the pixel conversion, and determining the completed handwritten content as the handwritten content in the user handwriting area;

[0166] If not, the fixed information area is cropped in the image module, and the remaining content in the image module is determined as the handwritten content in the user handwriting area.

[0167] Based on the above embodiments, an error prompt module may be further included to:

[0168] When it is determined that the recognition result of the target first information item does not match the record value of the target first information item in the target information system, calculating a similarity between the record value of the target first information item and the recognition result;

[0169] When the similarity is greater than a preset similarity threshold, generating correction prompt information of the target first information item according to the recorded value;

[0170] Otherwise, an error prompt of the target first information is directly generated.

[0171] The handwritten document processing device provided in the embodiment of the present invention can execute the handwritten document processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0172] Example 4

[0173] Figure 8 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0174] like Figure 8 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0175] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0176] The processor 11 can be various general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the handwritten document processing method described in any embodiment of the present invention. That is:

[0177] Collect handwritten document images and perform standardization processing on the handwritten document images to obtain standard handwritten documents;

[0178] Determine the target document template, target information system, and application page based on the standard handwritten document;

[0179] According to the target document template, the handwritten content of each information item in the standard handwritten document is recognized to obtain the recognition result of each information item;

[0180] The recognition result of the first information item is checked according to the target information system, and after the recognition result of the first information item is checked to be correct, the application page is filled with data according to the recognition results of each information item.

[0181] In some embodiments, the handwritten document processing method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the handwritten document processing method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to execute the handwritten document processing method in any other suitable manner (e.g., via firmware).

[0182] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0183] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0184] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0185] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0186] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0187] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0188] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0189] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for processing handwritten documents, characterized in that: include: Collect handwritten document images and perform standardization processing on the handwritten document images to obtain standard handwritten documents; Determine the target document template, target information system, and application page based on the standard handwritten document; According to the target document template, the handwritten content of each information item in the standard handwritten document is recognized to obtain the recognition result of each information item; The recognition result of the first information item is checked according to the target information system, and after the recognition result of the first information item is checked to be correct, the application page is filled with data according to the recognition results of each information item.

2. The method according to claim 1, characterized in that Standardize handwritten document images to obtain standard handwritten documents, including: Use the median filter algorithm to update the pixel value of each pixel in the handwritten document image; Binarize each pixel according to the pre-selected global threshold and the updated pixel value of each pixel; Corner points and edge feature points of the handwritten document image are extracted, and tilt correction is performed on the handwritten document image based on the corner points and edge feature points.

3. The method according to claim 1, characterized in that Based on the standard handwritten document, determine the target document template, target information system, and application page, including: Using optical character recognition (OCR) technology to recognize a standard handwritten document, obtaining a first recognition result, and determining a target document template based on the first recognition result and a pre-established document identification library; Determine the target information system and application page based on the target document template.

4. The method according to claim 1, wherein The target document template includes the target document base, user handwriting areas and fixed information areas pre-divided in the target document base, and information items corresponding to the user handwriting areas; Based on the target document template, the handwritten content of each information item in the standard handwritten document is recognized, and the recognition results of each information item are obtained, including: Determining, in a standard handwritten document according to the target document template, a plurality of user handwriting areas and at least one fixed information area corresponding to each user handwriting area; Segment the standard handwritten document based on the user's handwritten area, obtain multiple segmented image modules, and perform any of the following: When the user handwriting area belongs to the information selection type, the association between the handwriting content and each option is identified in the image module, and the recognition result of the information item corresponding to the user handwriting area is obtained based on the association between the handwriting content and each option; When the user handwriting area belongs to the information filling type, the fixed information area is removed in the image module to obtain the handwritten content in the user handwriting area, and the handwritten content in the user handwriting area is subjected to OCR recognition. According to the OCR recognition result, the recognition result of the information item corresponding to the user handwriting area is obtained.

5. The method according to claim 4, characterized in that In the image module, the fixed information area is removed to obtain the handwritten content in the user handwriting area, including: Identify whether the handwritten content overlaps with the fixed information area; If so, identifying feature points of the fixed information area in the image module, performing pixel conversion on the feature points of the fixed information area, completing the handwritten content in the image module after the pixel conversion, and determining the completed handwritten content as the handwritten content in the user handwriting area; If not, the fixed information area is cropped in the image module, and the remaining content in the image module is determined as the handwritten content in the user handwriting area.

6. The method according to claim 1, characterized in that After verifying the recognition result of the first information item according to the target information system, the method further includes: When it is determined that the recognition result of the target first information item does not match the record value of the target first information item in the target information system, calculating a similarity between the record value of the target first information item and the recognition result; When the similarity is greater than a preset similarity threshold, generating correction prompt information of the target first information item according to the recorded value; Otherwise, an error prompt of the target first information is directly generated.

7. A handwritten document processing device, characterized in that: include: The handwritten document collection and processing module is used to collect handwritten document images and perform standard processing on the handwritten document images to obtain standard handwritten documents; A system information determination module, configured to determine a target document template, a target information system, and an application page based on the standard handwritten document; A handwritten content recognition module is used to recognize the handwritten content of each information item in a standard handwritten document according to a target document template and obtain recognition results for each information item; The data checking and filling module is used to check the recognition result of the first information item according to the target information system, and after the recognition result of the first information item is checked to be correct, fill in the data on the application page according to the recognition results of each information item.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to execute the handwritten document processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the handwritten document processing method according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the handwritten document processing method according to any one of claims 1 to 6.

Images