Voucher information processing apparatus, voucher information processing method, and program

Abstract

To provide a voucher information processor for reducing the labor of a user at the time of detecting a voucher, and for shortening a time required until the reading completion of voucher contents.SOLUTION: The voucher information processing apparatus 1 includes an image acquisition section 111 that acquires a first image including vouchers, a voucher position detection section 112 that detects a plurality of vouchers including a first voucher and a second voucher in the first image, an image cutout section 115 that cuts out a second image including the first voucher from the first image, an image processing section 116 that processes a region of the second voucher from the second image so that characters cannot be recognized when the first voucher and the second voucher overlap in the second image, and an image output section 118 that outputs the second image processed by the image processing section 116.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a documented information processing device, a documented information processing method, and a program. [Background technology]

[0002] To reduce paper handling costs and facilitate information sharing, electronic filing systems have been put into practical use. These systems digitize paper documents, such as receipts and application forms, using scanners and manage them as image data (also known as digitized documents). In these systems, paper documents are scanned and converted into digitized documents. The text information on the imaged documents is then extracted as code data using OCR (Optical Character Recognition) technology, and used as computer-processed data to support data collection and analysis.

[0003] The processing of supporting documents such as receipts and invoices is an example of the electronic filing system described above. For example, a certified public accounting firm or a tax accountant's firm (hereinafter simply referred to as an "accounting firm") receives original supporting documents such as receipts and invoices from clients, which are the basis for accounting processing, and processes the client's accounting. In this case, the original supporting documents (hereinafter simply referred to as "supporting documents" or "receipts") are converted into electronic data using a scanner or a smartphone camera and then handed over to the accounting firm. This image data of the original supporting documents is later used by accounting firm staff to input accounting data or for automatic reading using OCR processing.

[0004] Patent Document 1 discloses a method for acquiring a voucher image and an accounting processing system. This accounting processing system includes a voucher image extraction means that reads image data containing original vouchers such as one or more receipts or invoices and extracts the voucher portion from the image data, a shaping means that corrects the tilt and distortion of the extracted voucher image and shapes it into a rectangle, and a voucher image registration means that saves the shaped voucher image. In normal mode, the voucher image extraction means attempts to extract the voucher image from the image data using a first set of receipt extraction thresholds, and if the image data voucher image cannot be extracted using the first set, it attempts to extract the voucher image from the image data using a second set of receipt extraction thresholds that has a variable range for the receipt extraction thresholds. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6539844 Summary of the Invention [Problem to be solved by the invention]

[0006] The system in Patent Document 1 detects each document by edge detection. For example, if two or more documents overlap, making edge detection difficult, the user must manually specify the presence of an edge by pointing or other means, which can make detecting the document time-consuming.

[0007] Furthermore, in the system of Patent Document 1, if it is difficult to read the document using OCR processing due to reasons such as the document being tilted, the system rotates the document image by 90 degrees and attempts to read it. For this reason, it may take time to rotate the document for reading, which may result in a longer time required for reading.

[0008] The present disclosure provides a documented evidence information processing device, a documented evidence information processing method, and a program that can reduce the user's effort when detecting a documented evidence and shorten the time required to complete reading of the contents of the documented evidence. [Means for solving the problem]

[0009] One aspect of the present disclosure is a document information processing device that includes an acquisition unit that acquires a first image including a document, a detection unit that detects the document in the first image, an extraction unit that, if the document detected by the detection unit contains multiple character areas including a first character area and a second character area with characters having different orientations, extracts a second image including the first character area and a third image including the second character area, and an output unit that outputs the second image and the third image.

[0010] One aspect of the present disclosure is a method for processing evidential information, comprising: acquiring a first image including evidential evidence; detecting the evidential evidence in the first image; if the detected evidential evidence contains multiple character areas including a first character area and a second character area in which the character orientations are different from each other, cutting out a second image including the first character area and a third image including the second character area; and outputting the second image and the third image.

[0011] One aspect of the present disclosure is a program that causes a computer to execute the above-described evidential information processing method. [Effects of the Invention]

[0012] According to the present disclosure, the burden on the user when detecting evidence can be reduced, and the time required to complete reading of the contents of the evidence can be shortened. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a block diagram of a voucher information processing system including a voucher information processing device according to an embodiment of the present disclosure. [Figure 2] 1 is a block diagram of an overlapping evidence processing unit in the evidence information processing device of embodiment 1. FIG. [Figure 3] 1 is a flowchart of a voucher information processing method executed by the voucher information processing device of the first embodiment. [Figure 4]1A and 1B are conceptual diagrams of the evidential information processing executed by the evidential information processing device of the first embodiment, in which FIG. 1A shows an image before processing, and FIG. 1B shows an image after processing. [Figure 5] 10A and 10B are conceptual diagrams of another example of the evidential information processing executed by the evidential information processing device of the first embodiment, in which (A) shows an image before processing, and (B) shows an image after processing. [Figure 6] FIG. 10 is a block diagram of an overlapping evidence processing unit in the evidence information processing device of the second embodiment. [Figure 7] FIG. 10 is a flow diagram of a method for processing evidence information executed by the evidence information processing device of the second embodiment. [Figure 8] FIG. 10 shows an example of an image of two separate vouchers. [Figure 9] 10A and 10B are conceptual diagrams of the evidential information processing executed by the evidential information processing device of the second embodiment, in which (A) shows an image before processing, (B) shows an image in an intermediate process, and (C) shows an image after processing. [Figure 10] FIG. 11 is a flow diagram of a voucher information processing method executed by the voucher information processing device of the third embodiment. [Figure 11] 10A and 10B are screens showing the processing results of the evidence information processing device of the third embodiment, and are screens displayed on the operation terminal, where (A) is the initial screen and (B) is a screen displaying images of two evidences side by side. [Figure 12] This is the screen that is displayed by checking the "Overlay" checkbox in FIG. 11(B). [Figure 13] 13 is a screen showing the processing results by the evidential information processing device of the fourth embodiment, and is a screen displayed on the operation terminal. [Figure 14] This screen is displayed when the warning information details button in FIG. 13 is pressed. [Figure 15] 13 is a screen showing the processing results by the evidential information processing device of the fifth embodiment, and is a screen displayed on the operation terminal. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed description of well-known matters or redundant description of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0015] (Configuration of the evidence information processing system) 1 is a block diagram of a voucher information processing system including a voucher information processing device according to an embodiment of the present disclosure. The voucher information processing system 200 includes a voucher information processing device 1, a plurality of operation terminals 50 (50A, 50B), a plurality of scanners 60 (60A, 60B), and a communication network 70 that communicatively connects these devices.

[0016] The operation terminal 50 is a terminal used by an applicant (an example of a user) who inputs the evidence or a processing person (e.g., an accounting person) (an example of a user) who processes the evidence. The operation terminal 50 is, for example, a PC (Personal Computer) or a tablet terminal. The operation terminal 50, like a PC, is equipped with a control unit, a communication unit, a memory unit, an operation unit, and a display unit. The control unit is, for example, configured by a processor. The memory unit includes, for example, a ROM (Read Only Memory) or a RAM (Random Access Memory). The operation unit may be a keyboard, a mouse, a touch panel, or the like. The display unit is, for example, a liquid crystal display device. The display unit may display, for example, images, data, information, etc. processed by the evidence information processing device 1. The user can check the displayed images, data, information, etc. The operation terminal 50A is a terminal used by a processing person. The operation terminal 50B is a terminal used by an applicant.

[0017] The scanner 60 is a device that reads paper documents, including receipts, invoices, application forms, delivery notes, contracts, and other supporting documents, and creates digitized documents. The data format of the digitized documents may be general image data. For example, bitmap format or JPEG format may be used. The scanner 60 may be, for example, a scanner with an auto-feeder mechanism, and may read standard-sized paper documents. The scanner 60 may be, for example, a stand-type scanner that reads images of originals placed on a desk, and may read non-standard-sized paper documents.

[0018] (Basic configuration of evidence information processing device) The evidence information processing device 1 takes in the image data of the evidence read by the scanner 60 and performs various processes on it. As a result, the evidence information processing device 1 can reduce the user's effort when detecting the evidence and shorten the time required to complete reading the contents of the evidence.

[0019] The evidence information processing device 1 includes a communication unit 30, an image processing unit 10, an evidence information management unit 20, and a storage unit 40. The storage unit 40 includes a registered evidence database (DB) 40D.

[0020] The communication unit 30 communicates various data or information. The communication method used by the communication unit 30 may be wired communication or wireless communication, and may include communication methods such as a local area network (LAN), a wide area network (WAN), power line communication, or cellular communication for mobile phones (e.g., 5G, LTE (Long Term Evolution)). The communication unit 30 functions as a receiving unit that receives various data (e.g., image data) transmitted from the operation terminal 50, the scanner 60, or the like via the communication network 70. The communication unit 30 functions as a transmitting unit that transmits various data (e.g., image data) to the operation terminal 50, the scanner 60, or the like via the communication network 70. The data received by the communication unit 30 is processed by, for example, the image processing unit 10. The communication unit 30 transmits data processed by, for example, the image processing unit 10.

[0021] The image processing unit 10 is configured by, for example, a processor. The processor may include an MPU (Micro Processing Unit), a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or a GPU (Graphical Processing Unit). The processor may include various integrated circuits (for example, an LSI (Large Scale Integration) or an FPGA (Field Programmable Gate Array)). The image processing unit 10 realizes various functions by executing programs stored in the memory unit 40. The image processing unit 10 comprehensively controls each unit of the evidence information processing device 1 and performs various processes.

[0022] The image processing unit 10 has, as its functional configuration, an overlapping evidence processing unit 11, an OCR processing unit 12, a duplicate registration confirmation processing unit 13, an input support processing unit 14, and an input schedule processing unit 15. The image processing unit 10 is, for example, a part that controls the main functions of the evidence information processing device 1, and when the image data received by the communication unit 30 via the communication network 70 is image data of an evidence, it performs the processing described below.

[0023] The overlapping evidence processing unit 11 performs the processing described below when the image data of the acquired evidence is image data in which two or more sheets of evidence are overlapped (Embodiment 1). Also, the overlapping evidence processing unit 11 performs the processing described below when the image data of the acquired evidence is image data including two or more character areas with different character orientations (Embodiment 2). The overlapping evidence processing unit 11 will be described in detail later.

[0024] The OCR processing unit 12 recognizes the character or text portion of the image data processed by the overlapping evidence processing unit 11 and converts it into character data. The OCR processing unit 12 performs general OCR (Optical Character Reader) processing, that is, optical character recognition.

[0025] The duplicate registration confirmation processing unit 13 compares the image data newly processed by the overlapping voucher processing unit 11 with other image data, for example, image data of vouchers registered in the registered voucher database 40D, and determines whether they are the same image (same voucher). In this way, the duplicate registration confirmation processing unit 13 checks whether information about the same voucher has been registered twice. The input support processing unit 14 performs processing related to supporting the input of the voucher processor. The input schedule processing unit 15 performs processing related to prompting the applicant to input the scheduled voucher. Details of the duplicate registration confirmation processing unit 13, input support processing unit 14, and input schedule processing unit 15 will be described in embodiments 3 to 5.

[0026] The evidence information management unit 20 has a function for a user (applicant or processing person) to manage the evidence information processing device 1.

[0027] The memory unit 40 includes a primary memory device (e.g., RAM (Random Access Memory) or ROM (Read Only Memory)). The memory unit 40 may include a secondary memory device (e.g., HDD (Hard Disk Drive) or SSD (Solid State Drive)) or a tertiary memory device (e.g., an optical disk or SD card). The memory unit 40 may include other memory devices. The memory unit 40 may be detachable from the evidence information processing device 1. The memory unit 40 holds various data or information, etc.

[0028] The registered evidence database 40D stores images of evidence processed by the overlapping evidence processing unit 11, evidence information (first evidence information) regarding the contents of evidence obtained by character recognition from the image of evidence by the OCR processing unit 12, etc. Therefore, images of evidence and evidence information representing past evidence are stored. The registered evidence database 40D may also include original paper images obtained by scanning the original paper of the evidence. The images of the processed evidence, original paper images, and evidence information may be stored in association with each other. Users of the operation terminal 50 (applicants and processing personnel) can utilize the processed image data or evidence information, etc. registered in the registered evidence database 40D.

[0029] The supporting document information includes information for individually identifying the supporting document. The supporting document information includes supporting document identification information for identifying the supporting document. If the supporting document is an invoice, the supporting document identification information may include the invoice number, the amount, the transfer account number, etc. If the supporting document is a receipt, the supporting document identification information may include the date, the amount, the recipient or name, etc.

[0030] The registered evidence database 40D may be located outside the evidence information processing device 1, and may be, for example, an independent database server. The evidence information processing device 1 may be able to access the database server via the communication unit 30 and may be able to acquire the stored images, data, information, etc.

[0031] (Embodiment 1) Next, a description will be given of the evidence information processing device 1 according to embodiment 1. Fig. 2 is a block diagram of the overlapping evidence processing unit 11 in the evidence information processing device 1 according to embodiment 1. This embodiment is useful when the first image M1, which is image data received by the communication unit 30, includes images of two or more evidences.

[0032] In the illustrated example, the first image M1 includes a second image M2 including the first evidence V1 and a third image M3 including the second evidence V2 (before processing). The first evidence V1 and the second evidence V2 are different receipts (a receipt summary and its details), and the entire surface of the second evidence V2 is placed on top of the larger first evidence V1. The first evidence V1 and the second evidence V2 are tilted relative to each other, and the text on each is also tilted. The first image M1 may include not only the second image M2 and the third image M3, but also an image behind the two images, such as an image of the backing paper on which the first evidence V1 and the second evidence V2 are attached.

[0033] When a conventional OCR device reads such evidence, the orientation of the images is different, which may result in a reading error. However, in this embodiment, the overlapping evidence processing unit 11 performs the following processing on the first image M1, thereby outputting a second image M2 containing the first evidence V1 and a third image M3 containing the second evidence V2 separately (after processing). The overlapping evidence processing unit 11 also processes the processing area MX of the third image M3 in the second image M2 to make the characters unrecognizable. In this example, the area is blacked out. This allows the OCR processing unit 12 to properly recognize the characters or text portions in the second image M2 and the third image M3 processed by the overlapping evidence processing unit 11 and convert them into character data. The overlapping evidence processing unit 11 will be described in detail below.

[0034] In this embodiment, the overlapping evidence processing unit 11 includes an image acquisition unit 111, an evidence position detection unit 112, a magnitude relationship calculation unit 113, an overlap rate calculation unit 114, an image cropping unit 115, an image processing unit 116, an image rotation correction unit 117, and an image output unit 118.

[0035] The image acquisition unit 111 functions as an acquisition unit that acquires a first image M1 including evidence from the communication unit 30. The evidence position detection unit 112 functions as a detection unit that detects multiple evidences including a first evidence V1 and a second evidence V2 in the first image M1.

[0036] The document location detection unit 112 may use a trained detector as artificial intelligence to detect the multiple documents. When training the detector, the document location detection unit 112 generates and updates trained parameters and a trained model (detector) by running a learning program using a set of training data. The trained parameters are parameters (coefficients) adjusted through training. The trained model is a program incorporating the trained parameters. The document location detection unit 112 creates training data, for example, through annotation processing in the field of AI (artificial intelligence) learning. For example, the document location detection unit 112 inputs a large number of images containing any document, inputs the x- and y-coordinate values ​​of the four corners of the area where the document exists in the input images, and generates training data by digitizing the coordinated images. The input images may include images with and without multiple overlapping images. The voucher position detection unit 112 learns the presence or absence of a voucher in an arbitrary image, the position, size, and tilt of the voucher relative to the image, etc., based on the learning data. As a result, the voucher position detection unit 112 can detect the image (position) of each individual voucher, even if multiple images of vouchers overlap. The voucher position detection unit 112 can then detect the position, size, and tilt of the voucher relative to the image, etc. Note that the detector learning itself may be performed outside the voucher position detection unit 112.

[0037] The size relationship calculation unit 113 calculates the size relationship between the second image M2 including the first evidence V1 detected by the evidence location detection unit 112 and the third image M3 including the second evidence V2. The size relationship is expressed in terms of size or area, and in this example, the size relationship is M2>M3.

[0038] The overlap rate calculation unit 114 calculates the overlap rate between the second image M2 and the third image M3, i.e., the degree of overlap between the two images, from the second image M2 and the third image M3 detected by the document location detection unit 112 and the size relationship calculated by the size relationship calculation unit 113. The overlap rate calculation unit 114 can use, for example, IoU (Intersection over Union) as the overlap rate. IoU is the ratio of the area of ​​the overlapping portion to the total area.

[0039] The image cropping unit 115 functions as a cropping unit that crops out a second image M2 including the first evidence V1 from the first image M1. The image cropping unit 115 may crop out a third image M3 including the second evidence V2 from the first image M1.

[0040] When the first evidence V1 and the second evidence V2 overlap in the second image M2, the image processing unit 116 functions as a processing unit that processes the processing area MX corresponding to the area of ​​the second evidence V2 in the second image M2 to make character recognition impossible. In this example, the image processing unit 116 applies blackening to the processing area MX corresponding to the area of ​​the second evidence V2, thereby making character recognition in the processing area MX by OCR processing impossible. The processing to make character recognition impossible may be not only blackening, but also whitening, blurring, or other processing methods.

[0041] The image rotation correction unit 117 performs rotation correction on the second image M2 processed by the image processing unit 116. Rotation correction is a process of changing the orientation of the image of the document so that the document and its text face a predetermined direction (for example, the left-right direction of the paper surface of the drawing). The image rotation correction unit 117 may also perform rotation correction on the third image M3. For example, the image rotation correction unit 117 may perform rotation correction by detecting the up-down direction of the document contained in each image based on the characteristics of each image and adjusting the angle of each image based on the up-down direction of the document. The image rotation correction unit 117 may also perform rotation correction based on the direction of the text or the frame indicating the periphery of the document. The image rotation correction unit 117 can align the orientations of the second image M2 and the third image M3 by rotating the second image M2 and the third image M3.

[0042] The image output unit 118 functions as an output unit that outputs at least the second image M2 processed by the image processing unit 116 and the image rotation correction unit 117. The image output unit 118 may also output a third image M3.

[0043] Upon receiving the processing results from the image output unit 118, the OCR processing unit 12 functions as a recognition unit that recognizes characters in the second image M2 and the third image M3 based on general OCR processing. The OCR processing unit 12 recognizes the characters in each image and converts them into text data. This text data may be stored in the registered evidence database 40D, or may be transmitted to the operation terminal 50 via the communication unit 30 via the communication network 70. The duplicate registration confirmation processing unit 13 may also check the text data. The OCR processing unit 12 may also classify the evidence by type (e.g., receipt, invoice) based on the text data obtained by the OCR processing. The registered evidence database 40D may store image data, text data, or evidence information related to the evidence for each type of evidence.

[0044] 3 is a flow diagram of the evidential information processing method executed by the evidential information processing device 1 of embodiment 1. As a premise, for example, it is assumed that the communication unit 30 receives a first image M1 from the scanner 60 via the communication network 70. It is also assumed that the first image M1 includes one or more evidential documents.

[0045] The image acquisition unit 111 acquires a first image M1 from the communication unit 30 (step S1). The voucher position detection unit 112 detects the image of the voucher (voucher image, voucher area) in the first image M1 (step S2) and determines whether multiple images of the voucher exist in the first image M1 (step S3). If only one image exists in the first image M1 (step S3; No), the image cropping unit 115 crops out the image of this one voucher (step S4). The image rotation correction unit 117 rotates and corrects the cropped image of the voucher (step S5), and the image output unit 118 outputs the corrected image to the OCR processing unit 12 (step S6).

[0046] In step S3, if the voucher position detection unit 112 determines that multiple images exist in the first image M1 (step S3; Yes), the size relationship calculation unit 113 calculates the size relationship between the images of the two vouchers. Furthermore, the overlap rate calculation unit 114 calculates the overlap rate (degree of overlap) between the images of the two vouchers, i.e., the second image M2 corresponding to the first voucher V1 and the third image M3 corresponding to the second voucher V2. The overlap rate calculation unit 114 calculates, for example, the overlap rate between the first voucher V1 and the second voucher V2 in the second image M2. The overlap rate calculation unit 114 uses, for example, IoU (Intersection over Union) as the overlap rate.

[0047] If the overlap rate is less than a predetermined threshold (e.g., 0.075) (Step S7; No), the degree of overlap is small, and ignoring the overlapping portions is considered to cause little harm. Therefore, the image cropping unit 115 crops out images of the multiple vouchers, i.e., the second image M2 and the third image M3 (Step S8). The voucher information processing device 1 can reduce the processing load by omitting steps such as calculating the overlap rate. The image rotation correction unit 117 rotates and corrects the cropped second image M2 and third image M3 (Step S5), and the image output unit 118 outputs the corrected second image M2 and third image M3 to the OCR processing unit 12 (Step S6).

[0048] If the overlap rate is equal to or greater than a predetermined threshold in step S7 (step S7; Yes), the degree of overlap is large, and ignoring the overlapping portion is likely to result in actual harm, such as an OCR reading error. Therefore, the image processing unit 116 blacks out the processing area MX corresponding to the second evidence V2 in the large second image M2, thereby disabling character recognition in the processing area MX (step S9).

[0049] The image cropping unit 115 crops out a small third image M3 corresponding to the second evidence V2 that was present in the processing area MX (step S10). The image rotation correction unit 117 rotates and corrects the processed second image M2 and third image M3 (step S5), and the image output unit 118 outputs the corrected second image M2 and third image M3 to the OCR processing unit 12 (step S6).

[0050] FIG. 4 is a conceptual diagram of the evidence information processing executed by the evidence information processing device 1 of embodiment 1, where (A) shows an image before processing and (B) shows an image after processing. The second image M2 is an image of the first evidence V1, which is a receipt, and the third image M3 is an image of the second evidence V2, which is also a receipt. As explained above, the second image M2 has a larger image area than the third image M3. In particular, in this example, as shown in FIG. 4(A), the entire area of ​​the third image M3 overlaps the second image M2, and the third image M3 does not extend beyond the edges of the second image M2. Conventional edge detection techniques have difficulty detecting the second image M2 and the third image M3 from the first image M1.

[0051] The evidence information processing device 1 of this embodiment processes the unprocessed image shown in FIG. 4(A) as shown in FIG. 4(B) and outputs it. That is, the evidence information processing device 1 can detect the second image M2 and the third image M3 with high accuracy even when at least a portion of the second image M2 and the third image M3 overlap. Then, the evidence information processing device 1 processes the area in the second image M2 where the third image M3 exists into a processed area MX where character recognition is impossible. This allows the evidence information processing device 1 to remove areas in the second image M2 that may have character orientations different from those of the second image M2, thereby suppressing OCR reading errors of the second image M2 in the subsequent OCR processing.

[0052] Furthermore, the evidence information processing device 1 can also extract the third image M3. This allows the second image M2 and the third image M3, which are originally present in the first image M1, to be read separately. In addition, the evidence information processing device 1 performs rotation correction on the second image M2 and the third image M3 to align the character orientations of both images and adjust the character orientation to be suitable for OCR processing, thereby improving the reading accuracy of the OCR processing.

[0053] 5 is a conceptual diagram of another example of the evidential information processing executed by the evidential information processing device of embodiment 1, where (A) shows an image before processing and (B) shows an image after processing. The image before processing in FIG. 5(A) is the same as the image in FIG. 4(A). In this example, when the image cropping unit 115 crops out a small third image M3 corresponding to the second evidence V2 that existed in the processing area MX (step S10 in FIG. 3), the image processing unit 116 paints in black the processing area MX1 that exists around the third image M3 and corresponds to the second image M2, thereby making character recognition in the processing area MX1 impossible.

[0054] In the above example, the entire surface of the second evidence V2 is placed on top of the larger first evidence V1. However, this embodiment is applicable not only to such cases, but also to cases where a portion of the first evidence V1 and a portion of the second evidence V2 overlap each other. This embodiment is also applicable to cases where the first evidence V1 and the second evidence V2 are the same size, or where three or more evidence sheets overlap.

[0055] (Embodiment 2) Next, a description will be given of the evidence information processing device 1 according to embodiment 2. Fig. 6 is a block diagram of the overlapping evidence processing unit 11 in the evidence information processing device 1 according to embodiment 2. This embodiment is useful when the first image M1, which is image data received by the communication unit 30, includes multiple character areas with characters oriented in different directions.

[0056] In the illustrated example, the fourth image M4 containing the evidence includes a fifth image M5 containing a first character area C1 and a sixth image M6 containing a second character area C2 (before processing). The first character area C1 and the second character area C2 are character areas with different character orientations. In this example, the first character area C1 and the second character area C2 are adjacent to each other on the fourth image M4 corresponding to a single receipt, and the character orientations differ by 90 degrees. Note that the difference in character orientation may be other than 90 degrees. For example, the first character area C1 and the second character area C2 may correspond to the same third evidence V3 (receipt), but the character orientations may differ. The fourth image M4 may include not only the third evidence V3 but also an image behind the evidence, such as an image of the backing paper on which the third evidence V3 is attached.

[0057] When an OCR device reads a voucher in such a state, a reading error may occur. However, in this embodiment, the overlapping voucher processing unit 11 performs the following processing on the fourth image M4, thereby outputting a fifth image M5 including the first character area C1 and a sixth image M6 including the second character area C2 separately (after processing). This allows the OCR processing unit 12 to properly recognize the character or text portions of the fifth image M5 and the sixth image M6 processed by the overlapping voucher processing unit 11 and convert them into character data. The overlapping voucher processing unit 11 will be described in detail below.

[0058] In this embodiment, the overlapping evidence processing unit 11 includes an image acquisition unit 111, an evidence position detection unit 112, a character direction detection unit 119, an image cropping unit 115, an image rotation correction unit 117, and an image output unit 118.

[0059] The image acquisition unit 111 functions as an acquisition unit that acquires a fourth image M4 including the third evidence V3 from the communication unit 30. The evidence position detection unit 112 functions as a detection unit that detects the third evidence V3 in the fourth image M4.

[0060] The document location detection unit 112 may use a trained detector to detect the document. The method of training the detector may be the same as that described in the first embodiment.

[0061] The character direction detection unit 119 detects the character orientation of the third evidence V3 detected by the evidence position detection unit 112. Like the evidence position detection unit 112, the character direction detection unit 119 may use a trained detector as an artificial intelligence to detect the character orientation of the third evidence V3. During detector training, a learning program is executed using a set of training data to generate and update trained parameters and a trained model (detector). The character direction detection unit 119 creates training data, for example, through annotation processing in the field of AI learning. For example, the character direction detection unit 119 may input a large number of images containing any evidence, sorted by character orientation, input the x- and y-coordinate values ​​of the four corners of the area where the evidence is present in the input image, and generate training data by digitizing the coordinated images. In this case, the training data includes images of a single evidence image containing multiple character areas with different character orientations. The character direction detection unit 119 learns the direction (character direction) of one or more character areas contained in any of the above-mentioned images based on the learning data. Furthermore, the document position detection unit 112 also learns whether or not documents with different character directions are included in the same document. As a result, the document position detection unit 112 can detect the direction (angle) of each of multiple character areas present in the image. Note that the detector learning itself may be performed outside the document position detection unit 112.

[0062] Therefore, even if there are multiple areas in an image where the character areas have different directions in one document, the document position detection unit 112 and the character direction detection unit 119 can detect areas in an image where the character areas have different directions using a trained detector as artificial intelligence.

[0063] The third evidence V3 in this example includes two character areas with different character orientations, so the character orientation detection unit 119 may detect a first character area C1 and a second character area C2 with different character orientations.

[0064] The image cutout unit 115 functions as a cutout unit that cuts out a fifth image M5 including the first character region C1 and a sixth image M6 corresponding to the second character region C2 from the fourth image M4.

[0065] The image rotation correction unit 117 performs rotation correction on the fifth image M5 and the sixth image M6 cut out by the image cutout unit 115. Rotation correction is a process of changing the orientation of the image of the evidence so that the evidence and its text face a predetermined direction (for example, the left-right direction on the paper surface of the drawing). The image rotation correction unit 117 may perform rotation correction by, for example, detecting the up-down direction of the evidence contained in each image based on the characteristics of each image and adjusting the angle of each image based on the up-down direction of the evidence. The image rotation correction unit 117 may also perform rotation correction based on the direction of the text or the frame indicating the periphery of the evidence. Therefore, the image rotation correction unit 117 can align the orientations of the fifth image M5 and the sixth image M6 by rotating the fifth image M5 and the sixth image M6.

[0066] The image output unit 118 functions as an output unit that outputs the fifth image M5 and the sixth image M6 processed by the image rotation correction unit 117.

[0067] Upon receiving the processing results from the image output unit 118, the OCR processing unit 12 functions as a recognition unit that recognizes characters in the fifth image M5 and the sixth image M6 based on general OCR processing. The OCR processing unit 12 recognizes the characters in each image and converts them into text data. This text data may be stored in the registered evidence database 40D, or may be transmitted to the operation terminal 50 via the communication unit 30 via the communication network 70. The duplicate registration confirmation processing unit 13 may also check the text data. The OCR processing unit 12 may also classify the evidence by type (e.g., receipt, invoice) based on the text data obtained by the OCR processing. The registered evidence database 40D may store image data, text data, or evidence information related to the evidence for each type of evidence.

[0068] Fig. 7 is a flow diagram of the evidence information processing method executed by the evidence information processing device 1 of embodiment 2. In Fig. 7, for example, it is assumed that the communication unit 30 receives a fourth image M4 from the scanner 60 via the communication network 70. It is also assumed that the fourth image M4 includes one or more character areas of the evidence.

[0069] The image acquisition unit 111 acquires the fourth image M4 from the communication unit 30 (step S11). The document position detection unit 112 detects the document image (document image, document area) in the fourth image M4 (step S12). The character direction detection unit 119 detects the character direction of the third document V3 detected by the document position detection unit 112, and determines whether there are multiple character areas with different character directions, i.e., a first character area C1 and a second character area C2 (step S13).

[0070] If only one character area exists in the fourth image M4 (step S13; No), the image cutting unit 115 cuts out an image of the voucher corresponding to this one character area (step S14). The image rotation correction unit 117 rotates and corrects the cut-out image of the voucher (step S15), and the image output unit 118 outputs the corrected image to the OCR processing unit 12 (step S16).

[0071] In step S13, if the character direction detection unit 119 determines that multiple character regions with different character orientations exist in the fourth image M4 (step S13; Yes), the image clipping unit 115 separately clips out a fifth image M5 including the first character region C1 and a sixth image M6 including the second character region C2 (step S17). The image rotation correction unit 117 rotates and corrects the images of the voucher corresponding to the clipped fifth image M5 and sixth image M6 (step S15), and the image output unit 118 outputs the corrected images to the OCR processing unit 12 (step S16).

[0072] 8, there may be not only a fourth image M4 containing the third evidence V3, but also an image MY containing evidence VY separated from the third evidence V3. A fifth image M5 and a sixth image M6 are cut out from the fourth image M4, and the fifth image M5, the sixth image M6, and the image MY are output and subjected to OCR processing.

[0073] FIG. 9 is a conceptual diagram of the evidence information processing executed by the evidence information processing device 1 of embodiment 2, where (A) shows an image before processing, (B) an image in the intermediate process, and (C) an image after processing. The fourth image M4 is an image of the third evidence V3, which is a receipt. As explained above, as shown in FIG. 9(A), the fifth image M5 including the first character region C1 and the sixth image M6 including the second character region C2 are arranged adjacent to each other in the fourth image M4. The orientation of the characters in the first character region C1 and the orientation of the characters in the second character region C2 are mutually different by 90 degrees. Conventional methods such as edge detection make it difficult to detect the fifth image M5 and the sixth image M6 from the fourth image M4.

[0074] The evidence information processing device 1 of this embodiment cuts out a fifth image M5 and a sixth image M6 separately from the unprocessed image shown in Fig. 9(A) and outputs them as shown in Fig. 9(B). That is, the evidence information processing device 1 can detect the respective directions of the first character area C1 and the second character area C2, which have different character orientations, and can detect both the fifth image M5 and the sixth image M6, and can separately read both the fifth image M5 and the sixth image M6.

[0075] Furthermore, as shown in Figure 9 (C), the evidence information processing device 1 can perform rotation correction on the fifth image M5 and the sixth image M6, thereby adjusting the character orientation of both images to be aligned with a character orientation suitable for OCR processing, thereby improving the reading accuracy of OCR processing.

[0076] In the above example, two character regions exist in one evidence document. However, this embodiment is applicable not only to such a case, but also to a case where three or more character regions exist in one evidence document.

[0077] (Embodiment 3) When processing supporting documents, one of the major challenges is the review of the supporting documents submitted by the applicant. For example, it is important to review whether the submitted supporting document is inconsistent with past supporting documents that have already been processed (whether it is a duplicate of a past supporting document). It is also important to review whether the content of the submitted supporting document matches the original supporting document. As a typical example, the review of invoices, which are a type of supporting document, places a heavy burden on the person in charge of processing them in the accounting department of a company.

[0078] Until now, the above examination has been carried out by, for example, comparing the checksum to check whether the data on two supporting documents is identical. However, with this method, the data is not deemed identical unless it matches perfectly. For example, even if the data on two supporting documents is on the same original paper, differences in the timing of scanning by the scanner, differences in the model, etc., can cause the two documents to be recognized as different documents even if they actually contain the same data, which can reduce the accuracy of the examination.

[0079] The person in charge of processing the supporting documents must rely on memory to determine whether the newly submitted supporting documents are duplicates of previous supporting documents. In addition, when using an accounting system that manages both electronic data and paper documents, the person in charge of processing the supporting documents must visually check whether the submitted supporting document data matches the original supporting documents. This type of processing increases the burden on the person in charge of processing the documents and may lead to oversights during the review process.

[0080] Therefore, the evidence information processing device 1 of this embodiment supports the processing personnel in their work related to evidence. The evidence information processing device 1 compares evidence data (current evidence information) based on an image of the requested evidence (current evidence image) with evidence data (registered evidence information) based on an image of a previously processed evidence or an original paper image (registered evidence image) of the original paper of the evidence, and determines whether the evidence information matches. If they match, the evidence information processing device 1 issues a warning, and the processing personnel checks the evidence based on the warning information, thereby improving the accuracy of the examination and reducing the burden on the processing personnel.

[0081] The above process is mainly performed by, for example, the duplicate registration confirmation processing unit 13 in Fig. 1. The duplicate registration confirmation processing unit 13 receives the current evidence information related to the requested evidence information. The duplicate registration confirmation processing unit 13 compares the current evidence information with the registered evidence information registered in the registered evidence database 40D. If the duplicate registration confirmation processing unit 13 determines through the comparison that the two evidences overlap, the communication unit 30 transmits warning instruction information including an instruction to present warning information indicating the overlap to the operation terminal 50A via the communication network 70.

[0082] In the operation terminal 50A, the communication unit receives the warning instruction information, and the display unit etc. presents the warning information based on the warning instruction information. The person in charge of processing can check the warning information on the display unit of the operation terminal 50A. Note that the communication unit 30 of the evidential information processing device 1 may transmit the warning instruction information not only to the operation terminal 50A of the person in charge of processing, but also to the operation terminal 50B of the applicant.

[0083] Furthermore, when using an accounting system that manages paper documents along with electronic data, the operation terminal 50B acquires various information input by the applicant via the operation unit. The information input by the applicant is information related to the voucher that may be included in the current voucher information, and is also referred to as input voucher information. The operation terminal 50A may acquire and display the input voucher information input by the applicant. The person in charge of processing using the operation terminal 50 may also visually confirm whether the input voucher information matches the original voucher paper.

[0084] In this way, the person in charge of processing checks whether the current evidence information matches the registered evidence information through the judgment made by the evidence information processing device 1, and also visually checks the current evidence image and the original evidence paper. If both checks show a mismatch, that is, if the current evidence information is evidence information that has not yet been registered, approval is completed. Furthermore, the evidence information processing device 1 may also issue a warning if the input evidence information and the current evidence information are different.

[0085] Fig. 10 is a flow diagram of the evidential information processing method executed by the evidential information processing device 1 of embodiment 3. The flow of Fig. 10 may be performed after the flow of embodiment 1 of Fig. 3 or the flow of embodiment 2 of Fig. 7, for example. Note that the flow of Fig. 10 is not necessarily performed after the flow of Fig. 3 or Fig. 7, and can be performed independently of these flows.

[0086] The OCR processing unit 12 performs OCR processing on the image of the evidence (step S21). The image of the evidence to be subjected to OCR processing here is, for example, at least one of the second image M2, the third image M3, the fifth image M5, and the sixth image M6, which are output images in the first and second embodiments. Note that the image of the evidence may be an image of the evidence other than the image of the evidence obtained in the first or second embodiment. By the OCR processing, evidence information (current evidence information) is obtained from the image of the evidence.

[0087] The duplicate registration confirmation processing unit 13 determines whether or not the same evidence information (registered evidence information) has been registered in the registered evidence database 40D in the past (step S22) based on the current evidence information obtained by the OCR processing unit 12. In other words, the duplicate registration confirmation processing unit 13 functions as a determination unit that determines whether or not the evidence information registered in the registered evidence database 40D and the current evidence information are the same evidence information.

[0088] The duplicate registration confirmation processing unit 13, for example, compares the registered voucher identification information included in the registered voucher information with the current voucher identification information included in the current voucher information. If the registered voucher identification information and the current voucher identification information are the same, the duplicate registration confirmation processing unit 13 may determine that they are the same voucher information.

[0089] If the duplicate registration confirmation processing unit 13 determines that the registered evidence information and the current evidence information are the same evidence information, there is a high possibility that the image of the evidence in step S21 (the current evidence image) and the image of the evidence linked to the registered evidence information in the registered evidence database 40D (the registered evidence image) are duplicated. Therefore, the communication unit 30 transmits warning instruction information to the operation terminal 50A via the communication network 70 (step S23). The warning instruction information may include, for example, the warning information, the current evidence image, and the registered evidence image.

[0090] In the operation terminal 50A, the communication unit receives the warning instruction information, and the display unit displays the warning information. The display unit may also display the images of the two vouchers compared in step S22 (the current voucher image and the registered voucher image) together with the warning information (step S24). By checking this display, the person in charge of processing can easily visually determine whether the current voucher image and the registered voucher image match. Note that the operation terminal 50A makes one of the images of the two vouchers semi-transparent and displays the images of the two vouchers superimposed, allowing the person in charge of processing to easily recognize the differences between the images of the two vouchers and more easily visually confirm whether the vouchers are indeed duplicates.

[0091] Fig. 11 shows a screen of the processing results by the evidence information processing device 1 of embodiment 3, and is a screen displayed on the operation terminal 50A of the processing staff. The display unit of the operation terminal 50A displays the initial screen of Fig. 11(A). This initial screen includes warning information of "duplicate past invoice." When the processing staff presses the confirmation button for the evidence (invoice) on which this warning information is displayed, the display unit displays images of the two evidences (the current evidence image and the registered evidence image) side by side, as shown in Fig. 11(B).

[0092] FIG. 12 shows a screen displayed on the operation terminal 50A of the processor, in which one of the images of the two vouchers is made semi-transparent and the two images are superimposed. When the processor checks the "Superimpose" box in FIG. 11(B) via the operation unit of the operation terminal 50A, the display unit of the operation terminal 50A displays the screen shown in FIG. 12. In FIG. 12, the amount written in one of the images is "¥52,000," while the other is "¥53,000," and the other is the same. By checking the superimposed display as shown in FIG. 12, the processor can easily recognize that there is an error in the amount written in the amount section. Furthermore, the processor's evidence information processing device 1 may send the processing of the current evidence application back to the previous processor (e.g., the applicant) by having the processor operate the "Return" button in FIG. 11(B) via the operation unit.

[0093] (Embodiment 4) Generally, before submitting a voucher, an applicant who applies for (submits) the voucher uses the operation terminal 50B to input information for each item on the application form for the voucher via the operation unit of the operation terminal 50B, and transmits this information (inputted voucher information) to the operation terminal 50A of the person in charge of processing. The person in charge of processing may visually check the inputted voucher information against the image data of the voucher processed by the voucher information processing device 1. This work also places a heavy burden on the person in charge of processing.

[0094] In the evidence information processing device 1 of this embodiment, the communication unit 30 receives the application form data (input evidence information) from the applicant's operation terminal 50B via the communication network 70. The input support processing unit 14 determines whether or not predetermined items in the application form for the evidence match items in the evidence data previously read by the evidence information processing device 1. In other words, the evidence information processing device 1 determines whether or not predetermined information included in the input evidence information matches corresponding predetermined information included in the registered evidence information.

[0095] If the above-mentioned predetermined items match, the communication unit 30 transmits the matching registered evidence information to the operation terminal 50A. The communication unit of the operation terminal 50A receives this registered evidence information, and the display unit displays it as the initial value of the data in the application form for the evidence. Furthermore, if the above-mentioned predetermined information matches, the input support processing unit 14 may instruct the operation terminal 50A to perform a display that takes into account misrecognition of reading by the OCR processing unit 12 (for example, displaying similar information).

[0096] If the above-mentioned predetermined items do not match, the communication unit 30 may transmit warning instruction information including warning information to that effect to the operation terminal 50A. In the operation terminal 50A, the communication unit may receive the warning instruction information, and the display unit may display the warning information. The person in charge of processing may check the display of the warning information and perform processing such as new registration or correction of information including the item via the operation unit.

[0097] The above processing is mainly performed by, for example, the input support processing unit 14 in Fig. 1. The input support processing unit 14 receives an image of the OCR-processed voucher sent from the OCR processing unit 12. From the image of the applied voucher, the input support processing unit 14 identifies, for example, characters corresponding to a representative item, such as a transfer account number on an invoice. The input support processing unit 14 references the registered voucher database 40D and searches for the item linked to a registered, previously processed voucher, that is, the transfer account number in the registered voucher information.

[0098] The input support processing unit 14 determines whether or not there is registered evidence information in the registered evidence database 40D that has information that matches a representative item (e.g., a transfer account number) of the requested evidence. If there is, the communication unit 30 transmits the registered evidence information including the item to the operation terminal 50A of the processing staff via the communication network 70.

[0099] 13 is a screen showing the processing result by the evidential information processing device 1 of the embodiment 4, which is a screen displayed on the operation terminal 50A of the processing staff. In this example, the evidential document is an invoice.

[0100] In this example, the input support processing unit 14 compares the transfer account number of the invoice obtained by OCR processing with the transfer account number included in the registered evidence information registered in the registered evidence database 40D. If the comparison results in a match between the two transfer account numbers, the communication unit 30 transmits the registered evidence information including this transfer account number to the operation terminal 50A of the processing staff member via the communication network 70. The registered evidence information here includes basic information and detailed information about the evidence (see FIG. 13). In the operation terminal 50A, the communication unit receives the registered evidence information from the operation terminal 50A, and the display unit displays at least a part of this evidence information.

[0101] In addition, the input support processor 14 may generate a similar character string similar to the transfer account number obtained by the OCR process, taking into account misrecognition during the OCR process. In this case, the input support processor 14 generates a similar character string whose reliability when substituted for the character string of the transfer account number obtained by the OCR process is equal to or greater than a predetermined value. The input support processor 14 determines whether a transfer account number matching the generated similar character string is stored in the registered evidence database 40D instead of the character string of the transfer account number obtained by the OCR process. If a matching transfer account number is stored, the communication unit 30 may transmit the registered evidence information including this matching transfer account number to the operation terminal 50A together with the above-mentioned reliability information.

[0102] FIG. 13 shows a case where the two transfer destination account numbers are compared and the two transfer destination account numbers do not match. In this case, the input support processing unit 14 generates a similar string for the transfer destination account number obtained by OCR processing. The input support processing unit 14 references the registered evidence database 40D and acquires registered evidence information containing a transfer destination account number that matches the generated similar string. The communication unit 30 transmits warning instruction information to the operation terminal 50A. This warning instruction information may include warning information indicating that the two transfer destination account numbers do not match, the acquired registered evidence information, and the transfer destination account number obtained by OCR processing. The communication unit of the operation terminal 50A receives the warning instruction information, and the display unit displays, for example, the warning information, the acquired registered evidence information, and the transfer destination account number obtained by OCR processing.

[0103] The person in charge of processing can confirm that there is a high possibility that the applicant's input is incorrect by viewing the warning information W1 regarding the transfer account number on the screen of Figure 13. Furthermore, when the person in charge of processing presses the details button adjacent to the warning information W1 via the operation unit of the operation terminal 50A, the screen of Figure 14 is displayed. Figure 14 is an example of a screen displayed on the operation terminal 50A. Figure 14 allows the cause of the warning to be confirmed. The display of Figure 14 includes the transfer account number entered by the applicant (input content, for example, "2975936"), the transfer account number obtained by OCR processing (OCR result, for example, "2915936"), and similar transfer account numbers (similar candidates, for example, "2915935") registered in the registered evidence database 40D.

[0104] The person in charge of processing may check these three transfer account numbers, and if, for example, the transfer account number entered by the applicant is correct, may issue an instruction via the operation unit of the operation terminal 50A that the transfer account number entered by the applicant is correct. This instruction information may be transmitted to the evidence information processing device 1 via the communication unit. The communication unit 30 of the evidence information processing device 1 receives this instruction information, and since the transfer account number entered by the applicant is correct, the input evidence information including this transfer account number may be registered in the registered evidence database 40D.

[0105] The person in charge of processing may check these three transfer destination account numbers, and if, for example, the transfer destination account number entered by the applicant is incorrect, may instruct the correct transfer destination account number (for example, the OCR result or a similar candidate transfer destination account number) via the operation unit of the operation terminal 50A. This instruction information may be transmitted to the evidence information processing device 1 via the communication unit. The communication unit 30 of the evidence information processing device 1 may receive this instruction information, replace the transfer destination account number in the input evidence information with the correct transfer destination account number, and register the input evidence information in the registered evidence database 40D.

[0106] According to the fourth embodiment, for example, when a processing person determines the validity of data in an application form for a voucher, the evidence information processing device 1 compares the input evidence information with the OCR result or registered evidence information, etc., and can support the processing person in input work to check the application form based on whether or not these pieces of information match. Therefore, the evidence information processing device 1 can reduce the burden on the processing person in examining the application content.

[0107] Although the example mainly shows a case where the input supporting information is compared with the registered supporting information by determining whether the transfer account number matches, the present invention is not limited to this. For example, the input support processing unit 14 may search for registered supporting information using the invoice number, the transfer recipient, the transfer amount, the transfer date, or the like as a key. These items are usually information included in receipts.

[0108] Although it has been shown that the transfer destination account number is registered in the registered evidence database 40D, this is not limited to this. A transfer destination master database that stores information about transfer destinations may be provided separately from the registered evidence database 40D. The transfer destination master database may include, for example, information about the owner of the transfer destination account (name, address, telephone number, etc.), the transfer destination account number, etc. When the transfer destination master database manages information about transfer destinations, the above-mentioned correct transfer destination account number information may be reflected in the transfer destination master database.

[0109] (Embodiment 5) In the fifth embodiment, the operation terminal 50B used by an applicant who applies for (submits) a voucher may make a scheduled application to notify the applicant of the scheduled application before the application for the voucher. In the scheduled application, the operation terminal 50B inputs information indicating the scheduled content of each item on the application form for the voucher (scheduled voucher information) via the operation unit, and transmits this scheduled input voucher information to the voucher information processing device 1 via the communication unit. The scheduled input voucher information is information scheduled to be input as input voucher information and includes information similar to the voucher information (some of which becomes scheduled information). For example, in the case of an invoice, it includes the scheduled payment amount or the scheduled transfer destination. The scheduled input voucher information also includes contact information (e.g., email address, etc.) for the operation terminal 50B of the scheduled applicant. The control unit of the operation terminal 50B may cooperate with the registered voucher database 40D via the communication unit to input (generate) at least a portion of the scheduled input voucher information based on at least a portion of the registered voucher information.

[0110] The input schedule processing unit 15 of the evidence information processing device 1 receives the scheduled input evidence information from the operation terminal 50B via the communication unit 30 and registers it in the registered evidence database 40D. The input schedule processing unit 15 performs a process to prompt the submission of evidence (e.g., an invoice) based on the scheduled input evidence information. This allows the evidence information processing device 1 to prevent the failure to register or create evidence.

[0111] Here, at the stage of schedule application, the image of the evidence and the evidence information based on the OCR processing of the image of the evidence have not been registered. For example, when the schedule application is approved by the schedule applicant's superior, the input schedule processing unit 15 generates an input prompt notification that prompts the upload of the image of the evidence related to this schedule application. The input prompt notification includes, for example, a URL to which the data of the image of the evidence is uploaded. The communication unit 30 transmits the generated input prompt notification to the operation terminal 50B.

[0112] 15 is a screen showing the processing result by the evidential information processing device 1 of the fifth embodiment, which is a screen displayed on the operation terminal 50B of the prospective applicant. In this example, the evidential voucher is an invoice.

[0113] The schedule applicant checks the screen of FIG. 15 and refers to, for example, "Waiting for invoice registration" in the status on the screen. Then, for example, based on the schedule applicant's operation on the operation unit, the communication unit of the operation terminal 50B may send the image of the invoice to the specified location for upload. Therefore, the applicant who made the schedule application can easily upload the image of the invoice using the URL for uploading the invoice (evidence) data described above. The uploaded image of the evidence (invoice) is stored in the registered evidence database 40D in association with the scheduled input evidence information related to this schedule application. The scheduled input evidence information and the evidence images of the invoice, etc., are confirmed by the person in charge of processing at the scheduled time of processing (for example, the scheduled month of processing) and can be processed by the evidence information processing device 1 or the operation terminal 50A of the person in charge of processing, etc.

[0114] In addition, if there is an error in the applicant's scheduled application (such as an error in the transfer account number), as described in embodiment 4, the input support processing unit 14 may create warning information and send it to the operation terminal 50A and the operation terminal 50B.

[0115] According to embodiment 5, the evidence information processing device 1 has the above-mentioned scheduled claim input function, and can support the registration of evidence at a timing different from the timing of inputting the scheduled input evidence information by sending an input prompt notification to the scheduled applicant.

[0116] As described above, the evidence information processing device 1 of the above embodiment comprises an acquisition unit (e.g., image acquisition unit 111) that acquires a first image M1 including the evidence, a detection unit (e.g., evidence position detection unit 112) that detects multiple evidences including the first evidence and the second evidence in the first image M1, an extraction unit (e.g., image extraction unit 115) that extracts a second image including the first evidence from the first image M1, a processing unit (e.g., image processing unit 116) that, if the first evidence and the second evidence overlap in the second image, processes the area of ​​the second evidence (e.g., processing area MX) from the second image so that characters cannot be recognized, and an output unit (e.g., image output unit 118) that outputs the second image processed by the processing unit.

[0117] As a result, even if multiple (e.g., two) pieces of evidence are overlapped, the evidence information processing device 1 can detect each piece of evidence individually, for example, by using artificial intelligence. Therefore, even if multiple pieces of evidence are overlapped, especially if they completely overlap so that one piece of evidence is entirely contained in the other piece, the evidence information processing device 1 can detect each piece of evidence individually. As a result, the evidence information processing device 1 can reduce the burden on the user when detecting evidence and shorten the time required to complete reading the contents of the evidence.

[0118] The cropping unit may crop a third image including the second evidence from the first image M1. The output unit may output the third image. By cropping the images of each evidence included in the first image M1 separately, the evidence information processing device 1 can suppress a decrease in character recognition accuracy, for example, when OCR processing is performed on the cropped images.

[0119] Furthermore, the evidence information processing device 1 may include a recognition unit that recognizes characters in the second image and the third image. As a result, even when multiple evidences are overlapped, the evidence information processing device 1 can derive separate images including each of the multiple evidences, and by performing character recognition based on each derived image, it is possible to suppress a decrease in character recognition accuracy.

[0120] The second image may have a larger image area than the third image. This allows the evidence information processing device 1 to suppress a decrease in character recognition accuracy when multiple evidences are overlapped, even if the sizes of the images (image areas), i.e., the sizes of the multiple evidences, are different.

[0121] Furthermore, if the overlap rate between the first and second evidence documents in the second image is equal to or greater than a threshold, the processing unit may process the area of ​​the second evidence document from the second image to make the character unrecognizable. As a result, the evidence information processing device 1 performs the process of processing the area of ​​the second evidence document to make the character unrecognizable only when the overlap rate of multiple evidence documents is high and the probability of a character recognition error is high. Therefore, the evidence information processing device 1 can improve the character recognition accuracy of multiple evidence documents while reducing the processing load related to the processing process.

[0122] In addition, the evidential information processing device 1 of the above embodiment includes an acquisition unit that acquires a fourth image including the evidential document, a detection unit that detects the evidential document in the fourth image, an extraction unit that, if the evidential document detected by the detection unit contains multiple character areas including a first character area and a second character area with character orientations different from each other, extracts a fifth image including the first character area and a sixth image including the second character area, and an output unit that outputs the fifth image and the sixth image.

[0123] As a result, even if a single document contains multiple character areas with different character orientations, the document information processing device 1 can detect each character area individually using artificial intelligence, etc. Therefore, by including each character area in a separate image and performing character recognition, the character recognition accuracy of characters included in multiple character areas can be improved. As a result, the document information processing device 1 can reduce the burden on the user when detecting the document and shorten the time required to complete reading the contents of the document.

[0124] Furthermore, the evidence information processing device 1 may include a correction unit (for example, an image rotation correction unit 117) that performs rotation correction on the fifth image and the sixth image separately. The output unit may output the rotation-corrected fifth image and sixth image.

[0125] As a result, the evidential information processing device 1 can easily perform character recognition by rotating and correcting the fifth image and the sixth image so that the orientation of characters included in multiple character regions is aligned. Therefore, the evidential information processing device 1 can improve the accuracy of character recognition. Furthermore, by suppressing the occurrence of character recognition errors, it becomes unnecessary to manually adjust the angle of the image of the evidential information in order to perform accurate character recognition. Therefore, the evidential information processing device 1 can reduce the user's effort and shorten the time required to complete character recognition.

[0126] Furthermore, the evidence information processing device 1 may include a recognition unit that recognizes characters in the fifth image and the sixth image. As a result, even if the orientations of characters in multiple character areas in the evidence are different, the evidence information processing device 1 can correct the angles of each image by deriving separate images including each of the multiple character areas, and output them as front images with no angle in the character strings, thereby suppressing a decrease in character recognition accuracy by performing character recognition based on each derived image.

[0127] Furthermore, the evidential information processing device 1 may include a storage unit (e.g., registered evidential database 40D) that stores first evidential information related to the evidential information. The evidential information processing device 1 may include a determination unit (e.g., duplicate registration confirmation processing unit 13) that determines whether the first evidential information stored by the storage unit and second evidential information obtained by character recognition of at least one of the second image, the third image, the fifth image, and the sixth image are the same evidential information. The evidential information processing device 1 may include a transmission unit (e.g., communication unit 30) that transmits an instruction to present warning information when it is determined that the first evidential information and the second evidential information are the same evidential information.

[0128] As a result, the evidential information processing device 1 can confirm the consistency between the second evidential information obtained this time and the first evidential information obtained and accumulated previously by sending an instruction to present warning information to, for example, the operation terminal 50A. As a result, the evidential information processing device 1 can prevent the same evidential information from being registered multiple times by the processing staff, thereby improving the accuracy of accounting processing.

[0129] Furthermore, the first supporting evidence information may include first supporting evidence identification information that identifies the supporting evidence. The second supporting evidence information may include second supporting evidence identification information that identifies the supporting evidence. If the first supporting evidence identification information and the second supporting evidence identification information are the same, the determination unit may determine that the first supporting evidence information and the second supporting evidence information are the same supporting evidence information.

[0130] As a result, the evidence information processing device 1 can determine whether the second evidence information obtained this time matches the first evidence information obtained in the past, using the evidence identification information as a key.

[0131] Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure. Furthermore, the components of the above-described embodiments may be combined in any manner without departing from the spirit of the disclosure.

[0132] In the above embodiments, the processor may be physically configured in any manner. If a programmable processor is used, the processing content can be changed by changing the program, thereby increasing the degree of freedom in processor design. The processor may be configured as a single semiconductor chip, or may be physically configured as multiple semiconductor chips. When configured as multiple semiconductor chips, each control in the above embodiments may be realized by a separate semiconductor chip. In this case, these multiple semiconductor chips can be considered to constitute a single processor. The processor may also be configured as a semiconductor chip and a component (such as a capacitor) having a different function. A single semiconductor chip may be configured to realize both the functions of the processor and other functions. Multiple processors may also be configured as a single processor. [Industrial Applicability]

[0133] INDUSTRIAL APPLICABILITY The present disclosure is useful for a documented evidence information processing device, a documented evidence information processing method, a program, etc., which can reduce the user's effort when detecting documented evidence and shorten the time required to complete reading of the documented evidence contents. [Explanation of symbols]

[0134] 1. Voucher information processing device 10 Image processing section 11 Overlapping evidence processing unit 12 OCR processing section 13 Duplicate registration confirmation processing unit 14 Input support processing unit 15 Input schedule processing section 20. Voucher Information Management Department 30 Communications Department 40 Storage section 40D Registration Voucher Database 50 Operation terminal 60 Scanner 70 Communication Network 111 Image acquisition unit 112 Evidence location detection unit 113 Size relationship calculation unit 114 Duplication rate calculation section 115 Image extraction section 116 Image Processing Department 117 Image rotation correction unit 118 Image output unit 200 Voucher Information Processing System C1 First character area C2 Second character area M1 First Image M2 2nd image M3 Third Image M4 4th Image M5 5th image M6 6th image MX processing area V1 First evidence V2 Second evidence

Claims

1. an acquisition unit that acquires a first image including the evidence; a detection unit for detecting the document in the first image; a cutting-out unit that cuts out a second image including the first character area and a third image including the second character area when the document detected by the detection unit contains a plurality of character areas including a first character area and a second character area having characters with different orientations; an output unit that outputs the second image and the third image; A document information processing device comprising:

2. a correction unit that performs rotation correction separately on the second image and the third image, the output unit outputs the rotation-corrected second image and the rotation-corrected third image. The evidence information processing device according to claim 1 .

3. a recognition unit that recognizes characters in the second image and the third image, The evidence information processing device according to claim 1 or 2.

4. a storage unit that stores first evidence information relating to the evidence; a determination unit that determines whether the first evidential information stored by the storage unit and the second evidential information obtained by character recognition of the second image or the third image are the same evidential information; If it is determined that the first evidence information and the second evidence information are the same evidence information, a transmitting unit that transmits an instruction to present the warning information, The evidence information processing device according to any one of claims 1 to 3.

5. the first evidence information includes first evidence identification information for identifying the evidence, the second evidence information includes second evidence identification information for identifying the evidence, the determination unit determines that the first evidence information and the second evidence information are the same evidence information when the first evidence identification information and the second evidence identification information are the same evidence information; The evidence information processing device according to claim 4.

6. obtaining a first image including the evidence; Detecting the document in the first image; When the detected evidence contains a plurality of character areas including a first character area and a second character area in which the character orientations are different from each other, cutting out a second image including the first character area and a third image including the second character area; outputting the second image and the third image; A method for processing evidence information.

7. A program that causes a computer to execute the evidential information processing method according to claim 6.

Images