Information processing apparatus, information processing method and computer-readable medium

Abstract

Information processing apparatus, information processing method, and computer-readable medium are provided. The information processing apparatus includes a processor that obtains a character recognition result for a first image of a read object, the read object having one or more first columns for filling in characters and a second column for signing or stamping, wherein the first and second columns are respectively filled in or stamped; if any of the first columns is defective, the processor generates a second image that makes the second column and the defective first column blank; if the second column is defective, the processor generates a second image that makes the second column blank, and outputs the generated second image.

Description

Technical Field

[0001] This invention relates to information processing apparatus, information processing method, and computer-readable medium. Background Technology

[0002] In recent years, a bill processing device has been proposed that generates a refillable bill when the content entered in the fill field of the bill does not meet the preset restrictions in the fill field. The refillable bill is provided with a fill field for refilling (for example, see Japanese Patent Application Publication No. 2016-53876).

[0003] The bill processing apparatus described in the aforementioned patent publication comprises: a first identification unit that identifies the content filled in the fill field from a bill image obtained by reading a bill with printed bill data and filling in the fill field; a generation unit that, if the identification result of the first identification unit does not meet the preset restrictions in the fill field, generates refill bill data including the refill fill field and additional information representing the refill bill, and prints it with the printing unit; and a second identification unit that, if additional information is detected in a bill image obtained by optically reading a bill printed by the printing unit and filled in the refill fill field, identifies the content filled in the refill fill field from the bill image.

[0004] In cases where a document requiring a signature or seal is defective, and assuming the originally submitted document is the original, it is necessary to establish and save the resubmitted document in advance with the original document. Summary of the Invention

[0005] The objective of this invention is to provide an information processing apparatus, an information processing method, and a computer-readable medium that, in the event that the object being read is defective, preserves the original and outputs a replacement image for resubmitting the defective object.

[0006] According to a first aspect of the present invention, an information processing apparatus is provided, wherein the information processing apparatus has a processor that obtains a recognition result of characters identified in a first image of a reading object, the reading object having one or more first columns for filling in characters and a second column for signing or stamping, the first column and the second column being filled in or stamped respectively, in the event that any of the first columns is defective, the processor forms a second image that makes the second column and the defective first column blank, and in the event that the second column is defective, the processor forms a second image that makes the second column blank, and outputs the formed second image.

[0007] According to a second aspect of the present invention, the processor outputs the second image by printing the second image on paper and outputting it.

[0008] According to a third aspect of the present invention, the processor reflects the recognition result into the defect-free first column to form the second image.

[0009] According to a fourth aspect of the present invention, the processor eliminates the first image after generating or outputting the second image.

[0010] According to a fifth aspect of the invention, the processor recognizes characters in the third image obtained by adding or stamping in the second image.

[0011] According to a sixth aspect of the invention, the processor outputs the second image in a manner that enables it to identify defective columns relative to undefective columns.

[0012] According to the seventh aspect of the invention, the manner is a way of representing the frame of the column.

[0013] According to the eighth aspect of the present invention, when the processor outputs the second image, it also outputs other images showing the defective parts.

[0014] According to a ninth aspect of the invention, the processor indicates defective columns in the first image by marking them.

[0015] According to a 10th aspect of the invention, the processor detects a rectangular region circumscribed to the mark and makes that rectangular region a blank column.

[0016] According to the eleventh aspect of the present invention, the first image is transmitted online.

[0017] According to the 12th aspect of the present invention, in the online transmission of the first image, the processor requires a file for personal authentication.

[0018] According to the 13th aspect of the invention, the processor displays the defective portion and, if permitted, forms the second image.

[0019] According to the 14th aspect of the present invention, the processor fills a pre-defined reading area with a background color to form the blank bar.

[0020] According to a 15th aspect of the present invention, a computer-readable medium is provided, which stores a program for causing a computer to perform processing, wherein, in the processing, a recognition result of recognizing characters in a first image of a reading object is obtained, the reading object having one or more first columns for filling in characters and a second column for signing or stamping, wherein the first column and the second column are respectively filled in or stamped; in the case that any of the first columns is defective, a second image is formed that makes the second column and the defective first column blank, and in the case that the second column is defective, a second image is formed that makes the second column blank; and the formed second image is output.

[0021] According to a 16th aspect of the present invention, an information processing method is provided, wherein: a recognition result of characters identified in a first image of a reading object is obtained, the reading object having one or more first columns for filling in characters and a second column for signing or stamping, wherein the first column and the second column are respectively filled in or stamped; in the case that any of the first columns is defective, a second image is formed that makes the second column and the defective first column blank; in the case that the second column is defective, a second image is formed that makes the second column blank; and the formed second image is output.

[0022] (Effect)

[0023] According to the first, 15, or 16th scheme, in the case of a defective read object, the original version can be guaranteed and a replacement image for the defective read object can be output.

[0024] According to the second scheme, it is possible to resubmit immediately at the window of the submission destination.

[0025] According to the third scheme, when re-identifying the blanks in column 1 that are free of defects, the recognition rate is higher than that of handwriting.

[0026] According to the fourth scheme, there is no need for the process of establishing an association between the first image and the second image.

[0027] According to the fifth scheme, originality can be guaranteed by making only the third image the original.

[0028] According to the sixth scheme, it is possible to distinguish the columns that need to be refilled from other columns.

[0029] According to the seventh solution, by simply changing the attribute information of the original frame, it is possible to make it a column that can be identified as defect-free.

[0030] According to the eighth scheme, it is possible to prepare for resubmission while observing other images.

[0031] According to the 9th or 10th scheme, the area that should be a blank column can be detected more accurately compared to the case where no marker is used.

[0032] According to the 11th scheme, it is possible to submit read objects online.

[0033] According to the 12th scheme, illegality can be suppressed compared to cases where documents do not require personal authentication.

[0034] According to the 13th scheme, if defects are tolerable, resubmission is not required.

[0035] According to the 14th scheme, it is possible to exclude parts with different background colors from the original version. Attached Figure Description

[0036] Figure 1 This is a block diagram illustrating an example of the schematic structure of an information processing system according to the first embodiment of the present invention.

[0037] Figure 2 This is a diagram showing an example of a negotiable instrument.

[0038] Figure 3 This is a diagram showing an example of a bill item information table.

[0039] Figure 4 This is an example of a ticket image.

[0040] Figure 5 This is a diagram showing an example of a table containing information about the ticket items that records the identification results.

[0041] Figure 6 This is an example of a ticket image used for resubmission.

[0042] Figure 7 This is a flowchart illustrating an example of the operation of an information processing device.

[0043] Figure 8 This is a block diagram illustrating an example of the schematic structure of an information processing system according to a second embodiment of the present invention.

[0044] Figure 9 This is an example of an image of a ticket containing markings.

[0045] Figure 10 This is a flowchart illustrating an example of the operation of an information processing device.

[0046] Figure 11 This is a diagram showing an example of the appendix of Modified Example 1. Detailed Implementation

[0047] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Furthermore, in each drawing, structural elements having substantially the same function are labeled with the same reference numerals, and repeated descriptions are omitted.

[0048] [Summary of Implementation Methods]

[0049] The information processing apparatus of this embodiment has a processor that obtains the recognition result of characters identified in a first image of a reading object. The reading object has one or more first columns for filling in characters and a second column for signing or stamping. The first column and the second column are filled in or stamped respectively. If any first column is defective, a second image is formed that makes the second column and the defective first column blank. If the second column is defective, a second image is formed that makes the second column blank. The formed second image is then output.

[0050] "Reading object" refers to the object from which characters are read, including paper documents and electronic documents containing handwritten characters. Examples include application forms, invoices, delivery notes, contracts, and specifications. Regarding "the first image of the reading object," when the reading object is a paper document, the image obtained by reading the paper document is the first image; when the reading object is an electronic document, the electronic document itself is the first image.

[0051] A "defect" refers to the failure to accurately perform the action of filling in or stamping a document. Defects can be formal or substantive. Formal defects include, for example, omissions of required fields (e.g., entering a number with fewer than 5 digits where a 6-digit account number should be), omissions of selected symbols (0, checkboxes, etc.), lack of a mark, or a mark protruding excessively from the stamping area. Substantive defects include, for example, errors in filling in information such as addresses, low recognition rates (e.g., low recognition rates due to faint characters, distorted characters, or parts of characters protruding from the box, e.g., below 90%), or marks that are too faint to be detected. However, a strict distinction between formal and substantive defects is not necessary.

[0052] Character "recognition" refers to converting handwritten or typeface characters contained in an image of a read object into character codes such as JIS codes. As a recognition method, for example, the character pattern of the object can be converted into a vector, the standard pattern corresponding to the character code can be converted into a vector, and the portion below a threshold, expressed by metrics such as the distance between the two vectors (e.g., Euclidean distance) or similarity (e.g., cosine similarity), can be used as the recognition result. Furthermore, character recognition can also utilize learning models acquired through machine learning, such as deep learning.

[0053] [First Implementation]

[0054] Figure 1 This is a block diagram illustrating an example of the schematic structure of an information processing system according to the first embodiment of the present invention.

[0055] The information processing system 1 includes: an image forming apparatus 2 that reads an image (hereinafter referred to as "document image") from a ticket; and an information processing apparatus 3 that performs character recognition processing on the ticket image read by the image forming apparatus 2. In this figure, one image forming apparatus 2 and one information processing apparatus 3 are shown respectively, but considering throughput, one or more of each may be used. A ticket is an example of an object to be read.

[0056] The image forming apparatus 2 and the information processing apparatus 3 are connected via a network 4. The network 4 can be, for example, a LAN (Local Area Network), a WAN (Wide Area Network), an intranet, or the Internet, and can be wired or wireless.

[0057] Image forming apparatus 2 and information processing apparatus 3 are installed, for example, at the window or reception desk of a bank, district government office, insurance company, etc., and operated by a person performing window services (hereinafter referred to as "operator"). Furthermore, the location of image forming apparatus 2 and information processing apparatus 3 is not limited to the above-mentioned locations, but can also be in other locations such as factories, offices, etc. In addition, image forming apparatus 2 and information processing apparatus 3 can also be installed in locations far apart from each other. The following example illustrates the case where image forming apparatus 2 and information processing apparatus 3 are installed at a bank window.

[0058] (Structure of an image forming apparatus)

[0059] The image forming apparatus 2 is a multifunction printer with functions such as scanning, printing, copying, email, and faxing. Alternatively, the image forming apparatus 2 may also have both scanning and printing functions.

[0060] The image forming apparatus 2 includes a control unit 20, a storage unit 21, an operation display unit 22, an image reading unit 23, an image forming unit 24, and a communication unit 25.

[0061] The control unit 20 consists of a processor such as a CPU (Central Processing Unit), interfaces, etc. The control unit 20 controls the execution of various functions by executing the programs stored in the storage unit 21.

[0062] The storage unit 21 consists of ROM (Read Only Memory), RAM (Random Access Memory), hard disk, etc., and stores information such as programs and ticket images.

[0063] The operation display unit 22 is used for inputting and displaying information. The operation display unit 22 is, for example, a touch panel display, which has a structure in which a touch panel is superimposed on a display such as an LCD.

[0064] The image reading unit 23 includes: an automatic paper feeder disposed on the document table; and a scanner that optically reads the image of the original document from the original document disposed on the document table or from the original document conveyed by the automatic paper feeder.

[0065] The image forming unit 24 generates a printed object by forming an image on paper supplied from the paper tray in a manner such as electrophotography or inkjet printing.

[0066] The communication unit 25 communicates with the information processing device 3 via the network 4.

[0067] (Structure of an information processing device)

[0068] The information processing device 3 has a control unit 30 that controls each component of the information processing device 3. The control unit 30 is connected to an input unit 32, which is implemented by a storage unit 31, a keyboard, a mouse, etc.; a display unit 33, which is implemented by a liquid crystal display, etc.; and a communication unit 34 that communicates with external devices such as an image forming apparatus 2 via a network 4.

[0069] The control unit 30 consists of a processor such as a CPU (Central Processing Unit), interfaces, etc. The control unit 30 executes the program 310 stored in the storage unit 31, functioning as a character recognition unit 300, a mark detection unit 301, a defect determination unit 302, and a resubmission ticket generation unit 303. The individual units 300 to 303 will be described below.

[0070] Storage unit 31 consists of ROM (Read Only Memory), RAM (Random Access Memory), hard disk, etc., and stores program 310 and ticket item information table 311 (see reference). Figure 3 ), address database 312, bank database 313, recognition dictionary 314, ticket image 10b (refer to) Figure 4 ), and resubmitted ticket images 10c and 10d (refer to) Figure 6Various information such as [information missing]. Image 10b is an example of the first image. Image 10c, a resubmission document, is an example of the second image. Image 10d, a resubmission document obtained by filling in and stamping image 10c, is an example of the third image.

[0071] A Japanese address is registered in address database 312. Defect assessment unit 302 determines that the address is also accurately filled in the abbreviation of the street name / block number / house number displayed in the address. For example, if the address is displayed as "○○2-chome 11-6", it is determined that the abbreviation "○○2-11-6" is also accurately filled in.

[0072] The names of banks and their branches existing in Japan are registered in the bank database 313.

[0073] The dictionary 314 stores standard patterns, for example, according to each JIS code or other character code.

[0074] Figure 2 This is a diagram illustrating an example of a negotiable instrument. The instrument 10a shown is printed on paper and includes fields for the user to fill in: address 11a, bank name 11b, branch name 11c, account number 11d, and account name 11e (hereinafter also referred to as "Field 11"). It also includes fields for the user to sign and stamp: signature 12a and stamp 12b (hereinafter also referred to as "Field 2"). In this diagram, the dashed lines are not actually recorded on the instrument 10a, but rather represent a pre-defined rectangular reading area 13. The reading area refers to the region where the character patterns filled in the reading area 13 are converted into character codes.

[0075] Figure 3 This diagram illustrates an example of a ticket item information table 311. The ticket item information table 311 is stored in storage unit 31 according to the management number assigned to each user (A001 in this case). The ticket item information table 311 has multiple items such as "Read Item," "Item Category," "Identification Result," and "Read Coordinate Information." The read item, item category, and read coordinate information in the ticket item information, which consists of these items, are preset according to the type of ticket.

[0076] The "Read Items" section records the item names in columns 1 (11) and 2 (12). The address, name, stamp, bank name, branch name, account number, and account name of the read item correspond to column 11a (address), column 12a (signature), column 12b (stamp), column 11b (bank name), column 11c (branch name), column 11d (account number), and column 11e (account name), respectively.

[0077] The "Item Category" records any one of the following categories: Fill in, Signature, or Imprint. Fill in refers to the item category for fields other than Signature: Address 11a, Bank Name 11b, Branch Name 11c, Account Number 11d, and Account Name 11e. Signature refers to the item category for Signature field 12a. Imprint refers to the item category for Seal field 12b.

[0078] The "Recognition Results" section records the recognition results for characters and the detection results for marks. That is, if the item category is filled in, the string is recorded as the recognition result; if the item category is a mark, the mark is recorded as the recognition result.

[0079] The "Read Coordinate Information" section records the coordinate information (top left and bottom right coordinates) of the reading area of ​​the rectangle set for each reading item. In this figure, a1 and b1 represent the top left coordinates (x, y) of the rectangle, and c1 and d1 represent the bottom right coordinates (x, y) of the rectangle.

[0080] Next, the components 300 to 303 of the control unit 30 will be described.

[0081] The character recognition unit 300 refers to the reading coordinate information in the ticket item information table 311 to extract the reading region 13 from the ticket image 10b and the ticket image 10d for resubmission. It then performs recognition processing on the characters filled in the reading region 13 and records the recognition result in the "Recognition Result" field of the ticket item information table 311. The recognition processing includes, for example, converting the filled-in character pattern into a vector, converting the standard pattern corresponding to the character code registered in the recognition dictionary 314 into a vector, and using the character code whose evaluation value, expressed as the distance or similarity between the two vectors, is below a threshold as the recognition result.

[0082] The Imprint Detection Unit 301 uses the coordinate information read from the document item information table 311 to extract the red portion from the document image 10b and the resubmitted document image 10c to detect the imprint, and records it in the "Recognition Result" item of the document item information table 311.

[0083] The defect determination unit 302 compares the recognition results of the characters entered in the address field 11a with the address database 312. If no recognition result is registered in the address database 312, the address field 11a is determined to be a defective area. Similarly, for the characters entered in the bank name field 11b and the branch name field 11c, the recognition results are compared with the bank database 313. If no recognition result is registered in the bank database 313, either the bank name field 11b or the branch name field 11c is determined to be a defective area. Furthermore, the defect determination unit 302 can also use a bibliographic dictionary or similar method for the name in the signature field 12a. It can also use the corresponding databases for the account number field 11d and the account name field 11e.

[0084] If any column 11a-11e in column 11 is defective, the resubmission ticket generation unit 303 generates a resubmission ticket image 10c that makes column 2 12 and the defective column 11 blank. If column 2 12 is defective, a resubmission ticket image 10c that makes column 2 12 blank is formed. The formed ticket image 10c is sent to the image forming apparatus 2 and printed on paper, generating a resubmission ticket. The resubmission ticket image 10d obtained from the ticket filled out and stamped on the resubmission ticket becomes the reading object.

[0085] When generating the resubmission ticket image 10c, the defect-free first column 11 can remain handwritten or reflect the recognition result when generating the resubmission ticket image 10c. By using lead type reflecting the recognition result, a high recognition rate can be obtained when recognizing the resubmission ticket image 10d after the blank space has been filled in.

[0086] When making the defective columns 11 and 22 blank, the reading area 13 can be whitened to eliminate characters, or the background color used as the background color of the ticket 10a can be used to fill the reading area 13 to eliminate characters. By filling the reading area 13 with the background color, parts with different background colors can be excluded from the original.

[0087] (The actions of the information processing system)

[0088] Next, refer to Figures 4 to 7 Here is an example to illustrate the operation of information processing system 1. Figure 7 This is a flowchart illustrating an example of the operation of the information processing device 3.

[0089] The operator receives a completed and stamped ticket 10a from the user and places it on the document table of the image reading unit 23 of the image forming apparatus 2 to read the ticket image 10b from the ticket 10a. The image reading unit 23 reads the ticket image 10b from the ticket 10a using a scanner in color mode. The control unit 20 transmits the read ticket image 10b to the information processing apparatus 3 via the network 4 through the communication unit 25. Alternatively, an automatic paper feeder can be used when multiple tickets 10a are available.

[0090] Figure 4 This is an example of a bill image 10b. In bill image 10b, characters are handwritten in the address field 11a, bank name field 11b, branch name field 11c, account number field 11d, account name field 11e, and signature field 12a, and a red stamp is formed in the stamp field 12b.

[0091] The control unit 30 of the information processing device 3 receives the ticket image 10b sent from the image forming device 2 by the communication unit 34 and stores it in the storage unit 31 (S1).

[0092] The control unit 30 identifies the characters filled in from the ticket image 10b and detects the imprint (S2).

[0093] Specifically, the character recognition unit 300 refers to the reading coordinate information in the ticket item information table 311 to extract the reading region 13 from the ticket image 10b, and performs recognition processing on the characters filled in the reading region 13, such as... Figure 5 As shown, the identification results are recorded in the invoice item information table 311.

[0094] The imprint detection department 301 uses the coordinate information read from the document item information table 311 to extract the red portion from the document image 10b, and detects imprints, such as... Figure 5 As shown, the imprint is recorded in the document item information table 311. Alternatively, in the name recognition process, the imprint can be detected first, and the area to the left of the detected imprint can be used as the reading area for character recognition. Therefore, the reading area 13 can be omitted in the stamp column 12b.

[0095] Next, the defect determination unit 302 determines whether a part is defective (S3). For example, for the characters filled in the address field 11a, the recognition result is compared with the address database 312. If no recognition result is registered in the address database 312, the address field 11a is determined to be a defective part. In addition, for the characters filled in the bank name field 11b and the branch name field 11c, the recognition result is compared with the bank database 313. If no recognition result is registered in the bank database 313, the bank name field 11b or the branch name field 11c is determined to be a defective part.

[0096] If the part is determined to be without defects (S3: No), the process ends. If the process ends because the part is determined to be without defects, the ticket image 10b is the original. However, since the corresponding data is recorded in the ticket item information table 311, the control unit 30 can also delete the ticket image 10b from the storage unit 31.

[0097] If the part is determined to be defective (S3: Yes), the document is resubmitted and the document generation department 303 refers to the document item information table 311 to determine whether the item category of the defective part is filled in (S4).

[0098] If the error is determined to be filled in (S4: Yes), the reading area 13 of the defective part is resubmitted by the ticket generation unit 303, which fills in the background color (S5). That is, the characters filled in the reading area 13 of the ticket image 10b are filled in with the background color that is the background color of the ticket 10a, thereby eliminating the characters.

[0099] Next, the mark is removed by the document generation unit 303 (S6). That is, the red mark is removed from the document image 10b.

[0100] Next, the document generation unit 303 fills in the reading area 13 of the signature field 12a with the background color (S7). That is, the characters filled in the reading area 13 of the signature field 12a in the document image 10b are filled in with the background color that is the background color of the document 10a, thereby eliminating the signature.

[0101] Next, the resubmission ticket generation unit 303 replaces the defect-free parts of the ticket image 10b with the lead type of the recognition result recorded in the ticket item information table 311, so that the ticket image 10b becomes the resubmission ticket image 10c (S8).

[0102] The resubmission ticket generation unit 303 sends the resubmission ticket image 10c to the image forming apparatus 2 and issues a printing instruction (S9).

[0103] The image forming unit 24 of the image forming apparatus 2 prints the resubmitted ticket image 10c onto the ticket paper and outputs it.

[0104] Figure 6 This is a diagram showing an example of a resubmitted ticket image 10c. The resubmitted ticket image 10c shown in this diagram indicates a defect in the address field 11a. In the resubmitted ticket image 10c, the address field 11a is filled in with the background color, becoming a blank field, and the signature field 12a is also filled in with the background color, becoming a blank field. Furthermore, the stamp in the stamp field 12b has been erased.

[0105] The operator delivers a resubmission ticket to the user, which is printed with the resubmission ticket image 10c. The user receives the resubmission ticket, fills in their address in the blank address field 11a, fills in their name in the blank signature field 12a, stamps their seal in the erased stamp field 12b, and then delivers it to the operator.

[0106] The operator places the completed and stamped resubmission form provided by the user on the document table of the image reading unit 23 of the image forming apparatus 2 to read the resubmission form image 10d from the resubmission form. The image reading unit 23 reads the resubmission form image 10d from the resubmission form in color mode using a scanner. The control unit 20 sends the read resubmission form image 10d to the information processing device 3.

[0107] The control unit 30 of the information processing device 3 receives the ticket image 10d sent from the image forming device 2 via the communication unit 34, and stores the ticket image 10d in the storage unit 31 (S1).

[0108] Similar to the original ticket 10a, the control unit 30 extracts the reading area 13 from the ticket image 10d for resubmission, performs recognition processing on the characters filled in the reading area 13, records the recognition result in the ticket item information table 311, and detects the mark (S2).

[0109] Since the non-defective areas are marked with the identification results in lead type, they are identified with a high recognition rate. Therefore, they are basically determined to be areas without defects (S3: No), and the process ends.

[0110] Furthermore, if the processing ends when a defective area is determined to be without defects, the resubmitted ticket image 10d becomes the original. However, since the corresponding data is recorded in the ticket item information table 311, the control unit 30 can also delete the resubmitted ticket images 10c and 10d from the storage unit 31. Additionally, regarding defective areas, the previous recognition results can be used to recognize only the characters in the area being filled in this time.

[0111] In the case of a defective part (S3: Yes), proceed with steps S4 to S9 as described above.

[0112] [Second Implementation]

[0113] Figure 8 This is a block diagram illustrating an example of the schematic structure of an information processing system according to a second embodiment of the present invention. In the first embodiment, the defect determination unit 302 determines whether a defect exists; however, in this embodiment, the defect determination unit 302 is not used, and the defect detection unit 304 detects the defect. Hereinafter, the description will focus on aspects that differ from the first embodiment.

[0114] Similar to the first embodiment, the information processing device 3 includes a control unit 30, a storage unit 31, an input unit 32, a display unit 33, and a communication unit 34.

[0115] The control unit 30 consists of a processor such as a CPU (Central Processing Unit) and interfaces. The control unit 30 executes the program 310 stored in the storage unit 31 and functions as a character recognition unit 300, a mark detection unit 301, a resubmission ticket generation unit 303, and a mark detection unit 304, but does not function as a defect determination unit 302.

[0116] Similar to the first embodiment, the storage unit 31 stores the program 310 and the ticket item information table 311 (see reference). Figure 3 ), address database 312, bank database 313, recognition dictionary 314, ticket image 10b (refer to) Figure 4 ) and resubmitted ticket images 10c, 10d (refer to) Figure 6 (and other information.)

[0117] The mark detection unit 304 extracts a portion of a specific color from the ticket image 10b and detects mark 14. The color of mark 14 can be set by operating the input unit 32. The color of mark 14 is preferably a specific color (e.g., green, blue) other than the color used for filling (e.g., black) or the color used for printing (e.g., red). By using a specific color as the color of mark 14, it is easy to extract only mark 14 from the ticket image. In this embodiment, mark 14 is a line, but it can also be a closed shape such as a circle or rectangle, or other shapes.

[0118] The resubmitted ticket generation unit 303 uses the rectangular area circumscribed to mark 14 as the reading area 13, filling it with background color to create a blank space. That is, regarding the area where mark 14 is attached, the reading coordinate information from the ticket item information table 311 is not used; instead, the rectangular area circumscribed to mark 14 is used as the reading area 13. In this case, the operator can identify the defective area based on their judgment. Alternatively, a larger area than mark 14, even if it is not circumscribed to mark 14, can be used as the reading area 13. Therefore, characters can be eliminated even if mark 14 is not accurately attached.

[0119] (The actions of the information processing system)

[0120] Next, refer to Figure 9 and Figure 10 Here is an example to illustrate the operation of information processing system 1. Figure 10 This is a flowchart illustrating an example of the operation of the information processing device 3.

[0121] The operator receives the completed and stamped ticket 10a from the user and determines if there are any defects. When a defect is found, a mark 14 is affixed to the defective area, and the ticket 10a is placed on the document table of the image reading unit 23 of the image forming apparatus 2 to read the ticket image 10b from the ticket 10a. The image reading unit 23 reads the ticket image 10b from the ticket 10a using a scanner in color mode. The control unit 20 transmits the read ticket image 10b to the information processing apparatus 3 via the network 4 through the communication unit 25. Alternatively, an automatic paper feeder can be used when multiple tickets 10a are available.

[0122] Figure 9 This is an example diagram showing document image 10b. In document image 10b, characters are handwritten in the address field 11a, bank name field 11b, branch name field 11c, account number field 11d, account name field 11e, and signature field 12a. Furthermore, a red stamp is formed in the seal field 12b. Additionally, if the address is incorrect, a mark 14 is added to the address field 11a. Furthermore, for example, if there is doubt about the address, bank name, or branch name, the operator can input the characters written on the document into the input unit 32 and search the address database 312 and bank database 313 to determine if the corresponding characters exist.

[0123] The control unit 30 of the information processing device 3 receives the ticket image 10b sent from the image forming device 2 by the communication unit 34 and stores it in the storage unit 31 (S11).

[0124] The marker detection unit 304 determines whether there is a marker 14 of a specific color (S12). That is, as long as the portion of the specific color can be extracted from the ticket image 10b, the marker 14 is detected.

[0125] If mark 14 is not detected (S12: No), similar to the first embodiment, the control unit 30 identifies the filled-in character from the ticket image 10b, detects the mark, records the identification result and detection result in the ticket item information table 311 (S13), and ends the process. Even if the process ends because the area determined to be without mark 14 is not present, the ticket image 10b is the original. However, since the corresponding data is recorded in the ticket item information table 311, the control unit 30 can also delete the ticket image 10b from the storage unit 31.

[0126] If mark 14 is detected (S12: Yes), refer to the document item information table 311 to determine whether the item category of the part marked 14 is to be filled (S14).

[0127] If it is determined that the document has been filled in (S14: Yes), the document generation unit 303 uses the area outside the mark 14 as the reading area 13 and fills the reading area 13 with the background color (S15). That is, the characters and marks 14 filled in the reading area 13 in the document image 10b are filled in with the background color that is the background color of the document 10a, and the characters and marks 14 are eliminated.

[0128] Next, the mark is removed by the document generation unit 303 (S16). That is, the red mark is removed from the document image 10b.

[0129] Next, the document generation unit 303 fills in the reading area 13 of the signature field 12a with the background color (S17). That is, the characters filled in the reading area 13 of the signature field 12a in the document image 10b are filled in with the background color that is the background color of the document 10a, thereby eliminating the signature.

[0130] Next, the resubmission ticket generation unit 303 replaces the defective parts with the lead type of the recognition result recorded in the ticket item information table 311, so that the ticket image 10b becomes the ticket image 10c for submission (S18).

[0131] The control unit 30 sends the resubmitted ticket image 10c to the image forming apparatus 2 and issues a printing instruction (S19).

[0132] The image forming unit 24 of the image forming apparatus 2 will Figure 6 The image 10c of the resubmission ticket shown is printed on the ticket paper and output.

[0133] The operator delivers a resubmission ticket to the user, which is printed with the resubmission ticket image 10c. The user receives the resubmission ticket, fills in their address in the blank address field 11a, fills in their name in the blank signature field 12a, stamps their seal in the erased stamp field 12b, and then delivers it to the operator.

[0134] The operator places the completed and stamped resubmission form delivered by the user on the original document table of the image reading unit 23 of the image forming apparatus 2 to read the resubmission form image 10d from the resubmission form. The image reading unit 23 reads the resubmission form image 10d from the resubmission form using a scanner in color mode. The control unit 20 sends the read resubmission form image 10d to the information processing device 3.

[0135] The control unit 30 of the information processing device 3 receives the ticket image 10d sent from the image forming device 2 via the communication unit 34 and stores the ticket image 10d in the storage unit 31 (S11).

[0136] The control unit 30 determines whether a mark exists (S12). If there is no mark 14 (S12: No), then, similar to the original ticket 10a, the reading area 13 is extracted from the ticket image 10d for resubmission, the characters filled in the reading area 13 are recognized, the recognition result is recorded in the ticket item information table 311, and the mark is detected (S13). Since the recognition result is filled in the area without mark 14, the recognition rate is high, so the area is basically determined to be without defects, i.e., without mark 14 (S12: No).

[0137] In addition, if the processing is considered to have ended without marking 14, the control unit 30 may also delete the ticket images 10c and 10d for resubmission from the storage unit 31, similar to the first embodiment.

[0138] If marker 14 is present (S12: Yes), proceed with steps S14 to S19 as described above.

[0139] (Variation Example 1)

[0140] When generating the resubmission ticket image 10c, the resubmission ticket generation unit 303 also generates an attachment showing the defective parts and outputs the attachment from the image forming apparatus 2. The user can view the attachment while filling out and stamping it for resubmission.

[0141] Figure 11This is a diagram illustrating an example of an attachment. Attachment 10e includes message 15 and a resubmission ticket image 10c. The resubmission ticket image 10c addresses the column indicating the defect (in...). Figure 11 In the image, the address field 11a) is highlighted with a red accent line 16. The accent line 16 can also be a thicker line than other fields, a double line, or something else distinct from the undefective fields. The accent line 16 is an example of how a defective field can be identified relative to an undefective field. Alternatively, the frame can be omitted, and arrows or other markings can be added to indicate the defective area. Annex 10e shows an example of another image.

[0142] (Variation Example 2)

[0143] The control unit 30 can also send and receive ticket images 10b to 10d online between the user terminal and the information processing device 3. In this case, the ticket images 10b and 10d for resubmission can also be documents that the control unit 30 needs for personal authentication. The control unit 30 receives the ticket image 10b sent from the user terminal and the ticket image 10d for resubmission via the network 4 and stores them in the storage unit 31.

[0144] Documents used for identity verification, such as driver's licenses, insurance certificates, and payment cards, can also be received online. Submitting documents online is more difficult than verifying identity in person; however, with the help of documents for identity verification, it becomes easier compared to not having these documents. Additionally, Annex 10e of Variation 1 can also be sent online.

[0145] (Variation Example 3)

[0146] The control unit 30 can also display the ticket image 10b containing the defective portion on the display unit 33, and generate a ticket for resubmission with the operator's permission. There is also a possibility that the defect determination unit 302 may make an incorrect determination; a human can then make the judgment, thereby suppressing incorrect determinations.

[0147] (Variation Example 4)

[0148] In the above embodiments, a document with a stamp field is used; however, a document without a stamp field may also be used. In this case, if column 11 is defective, the defective column 11 and signature field 12a are made blank to generate a document for resubmission.

[0149] The embodiments of the present invention have been described above, but the embodiments of the present invention are not limited to the above embodiments and can be modified and implemented in various ways.

[0150] Each unit of the processor can also be composed of some or all of hardware circuits such as reconfigurable circuits (FPGA, Field Programmable Gate Array) and application-specific integrated circuits (ASIC).

[0151] In the above embodiments, processor refers to processor in a broad sense, including general-purpose processors (such as CPU, Central Processing Unit, etc.) and dedicated processors (such as GPU (Graphics Processing Unit), ASIC, FPGA, programmable logic devices, etc.).

[0152] Furthermore, the actions of the processor in the above embodiments can be performed not only by a single processor, but can be accomplished collaboratively by multiple processors located in physically separate positions. Moreover, the order of the processor's actions is not limited to the order described in the above embodiments and can be appropriately modified.

[0153] Furthermore, some structural elements of the above embodiments may be omitted or modified. Additionally, steps may be added, deleted, modified, or replaced during the process of the above embodiments. Furthermore, the program used in the above embodiments can be provided by recording it on a computer-readable recording medium such as a CD-ROM, or it can be pre-stored on an external server such as a cloud server and used via a network.

Claims

1. An information processing device, wherein, The information processing device has a processor that obtains the recognition result of characters identified from a first image of a read object. The read object has one or more first columns for filling in characters and a second column for signing or stamping, wherein the first column and the second column are filled in or stamped, respectively. In the event that any of the first columns is defective, the processor, regardless of whether the second column is defective, removes the signature or stamp in the second column from the second image, forming a second image that makes the second column and the defective first column blank. In the event that the second column is defective, the processor forms a second image that makes the second column blank and outputs the formed second image.

2. The information processing apparatus according to claim 1, wherein, The processor outputs the second image by printing it onto paper and then outputting it.

3. The information processing apparatus according to claim 1 or 2, wherein, The processor reflects the recognition result into the defect-free first column to form the second image.

4. The information processing apparatus according to claim 3, wherein, After generating or outputting the second image, the processor deletes the first image.

5. The information processing apparatus according to claim 4, wherein, The processor recognizes characters in the third image obtained by adding notes or stamping in the second image.

6. The information processing apparatus according to claim 1 or 2, wherein, The processor outputs the second image in a manner that allows it to identify defective columns relative to undefective columns.

7. The information processing apparatus according to claim 6, wherein, The method described is the way the box representing the column is displayed.

8. The information processing apparatus according to claim 6, wherein, When outputting the second image, the processor also outputs other images showing the defective areas.

9. The information processing apparatus according to claim 1 or 2, wherein, The processor identifies defective columns in the first image by marking them.

10. The information processing apparatus according to claim 9, wherein, The processor detects a rectangular area that is circumscribed by the marker and makes that rectangular area a blank column.

11. The information processing apparatus according to claim 1 or 2, wherein, The first image was sent online.

12. The information processing apparatus according to claim 11, wherein, In the online transmission of the first image, the processor requires a file for personal authentication.

13. The information processing apparatus according to claim 1 or 2, wherein, The processor displays the defective area and, with permission, forms the second image.

14. The information processing apparatus according to claim 1 or 2, wherein, The processor fills the pre-defined reading area with a background color to form the blank bar.

15. A computer-readable medium storing a program that causes a computer to perform processing, wherein... In the process, Obtain the recognition result of the first image of the reading object to identify the characters. The reading object has one or more first columns for filling in characters and second columns for signing or stamping. The first column and the second column are filled in or stamped respectively. In the event of a defect in any of the first columns, regardless of whether the second column is defective, the signature or seal in the second column is removed from the second image, forming a second image that makes both the second column and the defective first column blank; in the event of a defective second column, a second image that makes the second column blank is formed; and Output the resulting second image.

16. An information processing method, wherein, Obtain the recognition result of the first image of the reading object to identify the characters. The reading object has one or more first columns for filling in characters and second columns for signing or stamping. The first column and the second column are filled in or stamped respectively. In the event of a defect in any of the first columns, regardless of whether the second column is defective, the signature or seal in the second column is removed from the second image, forming a second image that makes both the second column and the defective first column blank; in the event of a defective second column, a second image that makes the second column blank is formed; and Output the resulting second image.

17. A computer program product comprising a program that causes a computer to perform processing, wherein, In the process, Obtain the recognition result of the first image of the reading object to identify the characters. The reading object has one or more first columns for filling in characters and second columns for signing or stamping. The first column and the second column are filled in or stamped respectively. In the event of a defect in any of the first columns, regardless of whether the second column is defective, the signature or seal in the second column is removed from the second image, forming a second image that makes both the second column and the defective first column blank; in the event of a defective second column, a second image that makes the second column blank is formed; and Output the resulting second image.

Images