Information processing device, information processing system, and information processing program

Abstract

To provide an information processing device, an information processing system, and an information processing program that can achieve both processing speed and character recognition accuracy in comparison with a case where single image-conversion processing is executed uniformly for an entire document as preprocessing for character recognition.SOLUTION: A preprocessing procedure determination unit 56 obtains information on: an attribute to be obtained that is predetermined according to a document type; preprocessing in obtaining the attribute in a document; and processing procedure. The unit then determines a procedure for the preprocessing. A preprocessing unit 58 executes the preprocessing for document information according to a result of the determination by the preprocessing procedure determination unit 56.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing system, and an information processing program. [Background technology]

[0002] Patent Document 1 proposes an image analysis device that extracts character information from a target image. The image analysis device includes an OCR engine trained by an OCR engine training device that includes a training image generator that generates training images by applying training image conversion to characters in a specific font, a training image generation training unit that trains the training image generator to perform training image conversion that converts a second image into a first image using a pair of a first image including a recognized character and a second image in which the recognized character is expressed in a specific font, and a character recognition training unit that trains the OCR engine to extract characters from an image using a pair of a training image generated by the training image generator and a character corresponding to the training image, and an OCR unit that extracts character information from a target image using the OCR engine.

[0003] Patent Document 2 proposes an image forming device that performs appropriate preprocessing when using an application provided by an external device. In detail, the image forming device determines preprocessing corresponding to the external application and registers the determined preprocessing in a memory. Then, when image processing using an external application is instructed, the image forming device passes data that has undergone preprocessing registered in the memory as corresponding to the external application to the external application. When determining the preprocessing, the image forming device performs a first image processing on the first image data to generate second image data, passes the second image data to the external application, and receives the processed data from the external application. Then, the image forming device determines whether or not the first image processing is the preprocessing corresponding to the external application based on the second image data and the processed data. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-82814 A [Patent Document 2] Patent No. 6237369 Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention aims to provide an information processing device, an information processing system, and an information processing program that can achieve both processing speed and character recognition accuracy, compared to when a single image conversion process is performed uniformly on an entire document as preprocessing for character recognition. [Means for solving the problem]

[0006] In order to achieve the above object, an information processing device according to a first aspect includes a processor, and the processor performs a predetermined image conversion process on a document to be subjected to character recognition as a preprocessing step for character recognition, for each attribute in the document or for each area in the document that is predetermined according to the type of the document, and performs character recognition on the document that has been subjected to the image conversion process, thereby outputting a result. The page order is changed so that the image conversion process is performed for each image conversion process. .

[0007] In addition, in an information processing device according to a second aspect, in the information processing device according to the first aspect, the image conversion process includes a first image conversion process that removes non-characters, and a second image conversion process that has lower character recognition accuracy than the first image conversion process but has a faster processing speed for removing non-characters.

[0008] In addition, an information processing device according to a third aspect is the information processing device according to the second aspect, wherein the first image conversion process is an image conversion process using artificial intelligence that has been trained in advance by machine learning.

[0009] In addition, in an information processing device according to a fourth aspect, in the information processing device according to the third aspect, the first image conversion process includes a plurality of types of image conversion process that differ for each target other than characters to be removed.

[0011] Also, 5The information processing device according to the embodiment is Any one of the first to fourth aspects In the information processing device according to the aspect, the processor executes the image conversion process by rearranging the pages in a predetermined page order for each type of document.

[0012] Also, 6 The information processing system according to the embodiment includes the first embodiment to the third embodiment. 5 The information processing device includes the information processing device according to any one of the aspects, and a requesting device that transmits a reading result of reading the document to the information processing device and requests character recognition.

[0013] Also, 7 The information processing program according to the embodiment causes a computer to execute, as a preprocessing step for character recognition, a predetermined image conversion process for each attribute in the document or for each area in the document that is predetermined according to the type of the document, and to execute character recognition for the document that has been subjected to the image conversion process, and to output a result. and executing the image conversion process by changing the page order so that the image conversion process is performed for each page. do. Effect of the Invention

[0014] According to the first aspect, it is possible to provide an information processing device that can achieve both high processing speed and high character recognition accuracy, compared to a case where a single image conversion process is uniformly performed on an entire document as preprocessing for character recognition. Moreover, compared to the case where the page order is not changed, the image conversion process can be performed more efficiently. do.

[0015] According to the second aspect, by using a plurality of types of image conversion processing, it is possible to achieve both high processing speed and high character recognition accuracy.

[0016] According to the third aspect, it is possible to improve the character recognition accuracy as compared with the case where another image conversion process is used as the first image conversion process.

[0017] According to the fourth aspect, it is possible to perform image conversion processing suited to the object to be removed other than characters.

[0019] No. 5According to this aspect, rearrangement of the page order becomes easier than rearranging the page order by determining a corresponding image conversion process for each page.

[0020] No. 6 According to this aspect, it is possible to provide an information processing system that can achieve both high processing speed and high character recognition accuracy, compared to a case in which a single image conversion process is uniformly performed on an entire document as preprocessing for character recognition.

[0021] No. 7 According to the aspect, it is possible to provide an information processing program that can achieve both high processing speed and high character recognition accuracy, compared to a case where a single image conversion process is uniformly performed on an entire document as preprocessing for character recognition. Moreover, compared to the case where the page order is not changed, the image conversion process can be performed more efficiently. do. [Brief description of the drawings]

[0022] [Figure 1] 1 is a diagram showing a schematic configuration of an information processing system according to an embodiment of the present invention. [Diagram 2] 2 is a block diagram showing a configuration of a main part of an electrical system of an image forming apparatus in the information processing system according to the embodiment; [Diagram 3] 2 is a block diagram showing the configuration of main electrical systems of a server, a mobile terminal, and a client terminal in the information processing system according to the embodiment; FIG. [Figure 4] FIG. 13 is a diagram for explaining extraction of attributes in a document that has been character-recognized. [Diagram 5] 2 is a functional block diagram showing a functional configuration of a server in the information processing system according to the embodiment. FIG. [Figure 6] An example of a list in which key words to be acquired for each type of document, the contents of preprocessing, and the processing positions thereof are predetermined is shown. [Figure 7] 10 is a flowchart showing an example of a flow of processing performed by a server of the information processing system according to the present embodiment. [Figure 8] An example of a list in which key words to be acquired for each type of document, the contents of preprocessing, and the processing order are predetermined is shown. [Figure 9]11A and 11B are diagrams for explaining an example in which the processing order is changed and pre-processing is executed; [Figure 10] 13 is a flowchart showing an example of a processing flow when pre-processing is performed by changing the page order of a contract in the server of the information processing system according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] An example of an embodiment of the present invention will be described in detail below with reference to the drawings. Fig. 1 is a diagram showing a schematic configuration of an information processing system according to this embodiment.

[0024] As shown in Fig. 1, the information processing system 10 according to the present embodiment includes an image forming device 12, a scanner device 13, a server 14 as an information processing device, a mobile terminal 15, and a client terminal 16. In the present embodiment, only one image forming device 12, one scanner device 13, one server 14, one mobile terminal 15, and one client terminal 16 are shown, but a plurality of each may be provided. The image forming device 12, the scanner device 13, the mobile terminal 15, and the client terminal 16 correspond to an example of a request device, and the others may be omitted except for at least one of them. In addition, for example, a personal computer is applied as the client terminal 16, and a mobile terminal such as a tablet terminal or a smartphone is applied as the mobile terminal 15.

[0025] The image forming device 12, the scanner device 13, the server 14, the portable terminal 15, and the client terminal 16 are each connected via a communication line 18 such as a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, an intranet, etc. The image forming device 12, the scanner device 13, the server 14, the portable terminal 15, and the client terminal 16 are each capable of transmitting and receiving various data to and from each other via the communication line 18.

[0026] FIG. 2 is a block diagram showing the configuration of a main part of an electrical system of the image forming apparatus 12 in the information processing system 10 according to this embodiment.

[0027] 2, the image forming apparatus 12 according to this embodiment includes a control unit 20 including a central processing unit (CPU) 20A, a read only memory (ROM) 20B, and a random access memory (RAM) 20C. The CPU 20A controls the overall operation of the image forming apparatus 12. The RAM 20C is used as a work area when the CPU 20A executes various programs. The ROM 20B stores various control programs, various parameters, and the like in advance. In the image forming apparatus 12, each component of the control unit 20 is electrically connected to each other via a system bus 42.

[0028] On the other hand, the image forming apparatus 12 according to the present embodiment includes a HDD (hard disk drive) 26 that stores various data, application programs, and the like. The image forming apparatus 12 is also connected to a user interface 22 and includes a display control unit 28 that controls the display of various operation screens, etc. on the display of the user interface 22. The image forming apparatus 12 is also connected to the user interface 22 and includes an operation input detection unit 30 that detects an operation instruction input via the user interface 22. In the image forming apparatus 12, the HDD 26, the display control unit 28, and the operation input detection unit 30 are electrically connected to a system bus 42. Note that, in the image forming apparatus 12 according to the present embodiment, an example in which the image forming apparatus 12 includes the HDD 26 will be described, but this is not limiting, and the image forming apparatus 12 may include a non-volatile storage unit such as a flash memory.

[0029] The image forming apparatus 12 according to the present embodiment includes a reading control unit 32 that controls the optical image reading operation by the document reading unit 46 and the document feeding operation by the document transport unit, and an image forming control unit 34 that controls the image forming process by the image forming unit 24 and the transport unit 25 to transport paper to the image forming unit 24. The image forming apparatus 12 also includes a communication line interface (communication line I / F) unit 36 ​​that is connected to the communication line 18 and transmits and receives communication data to and from other external devices such as the server 14 connected to the communication line 18, and an image processing unit 44 that performs various image processing. The image forming apparatus 12 also includes a facsimile interface (facsimile I / F) unit 38 that is connected to a telephone line (not shown) and transmits and receives facsimile data to and from a facsimile device connected to the telephone line. The image forming apparatus 12 also includes a transmission / reception control unit 40 that controls the transmission and reception of facsimile data via the facsimile interface unit 38. In the image forming apparatus 12, the transmission / reception control unit 40, the reading control unit 32, the image forming control unit , the communication line interface unit , the facsimile interface unit , and the image processing unit 44 are electrically connected to a system bus .

[0030] With the above configuration, the image forming apparatus 12 according to this embodiment executes access to the RAM 20C, the ROM 20B, and the HDD 26 by the CPU 20A. The image forming apparatus 12 also executes control of display of information such as an operation screen and various messages on the display of the user interface 22 via the display control unit 28 by the CPU 20A. The image forming apparatus 12 also executes control of operation of the document reading unit 46 and the document transport unit via the reading control unit 32 by the CPU 20A. The image forming apparatus 12 also executes control of operation of the image forming unit 24 and the transport unit 25 via the image formation control unit 34, and control of transmission and reception of communication data via the communication line interface unit 36 ​​by the CPU 20A. The image forming apparatus 12 also executes control of transmission and reception of facsimile data via the facsimile interface unit 38 by the transmission and reception control unit 40 by the CPU 20A. Furthermore, in the image forming apparatus 12, the CPU 20A grasps the operation content in the user interface 22 based on the operation information detected by the operation input detection unit 30, and executes various controls based on the operation content.

[0031] The scanner device 13 includes configurations similar to those of the control unit 20, the reading control section 32, and the document reading section 46 of the image forming device 12, and the basic configurations are similar, so detailed description thereof will be omitted.

[0032] Next, the main configuration of the electrical system of the server 14, the mobile terminal 15, and the client terminal 16 according to this embodiment will be described. Fig. 3 is a block diagram showing the main configuration of the electrical system of the server 14, the mobile terminal 15, and the client terminal 16 in the information processing system 10 according to this embodiment. Since the server 14, the mobile terminal 15, and the client terminal 16 are basically configured as general computers, the server 14 will be described as a representative, and the mobile terminal 15 and the client terminal 16 will be assigned only corresponding reference numerals and detailed description will be omitted.

[0033] As shown in FIG. 3, the server 14 according to the present embodiment includes a CPU 14A, a ROM 14B, a RAM 14C, a HDD 14D, a keyboard 14E, a display 14F, and a communication line interface (I / F) unit 14G. The CPU 14A controls the overall operation of the server 14. The ROM 14B stores various control programs and various parameters in advance. The RAM 14C is used as a work area when the CPU 14A executes various programs. The HDD 14D stores various data and application programs. The keyboard 14E is used to input various information. The display 14F is used to display various information. The communication line interface unit 14G is connected to the communication line 18 and transmits and receives various data to and from other devices connected to the communication line 18. The above-mentioned units of the server 14 are electrically connected to each other by a system bus 14H. Note that, in the server 14 according to the present embodiment, an example in which the server 14 includes the HDD 14D will be described, but the server 14 may include other non-volatile storage units such as flash memory.

[0034] With the above configuration, the server 14 according to the present embodiment executes, by the CPU 14A, access to the ROM 14B, the RAM 14C, and the HDD 14D, acquisition of various data via the keyboard 14E, and display of various information on the display 14F. In addition, the server 14 executes, by the CPU 14A, control of transmission and reception of communication data via the communication line interface unit 14G.

[0035] Incidentally, document management in companies is generally done by classifying documents by document type, company name, contract date, quotation date, etc., and managing them by filing, etc. When managing document contents centrally, document names, company names, main service names, dates, etc. are often transcribed separately into a spreadsheet program or the like, and managed so that they can be viewed at a glance. However, implementing such list management requires the work of bringing in files, opening the target files, searching for the places where the relevant contents are written, and transcribing while looking at them.

[0036] Therefore, in order to read a document and perform optical character recognition (OCR) processing to extract necessary information, in the information processing system 10 according to the present embodiment configured as described above, the server 14 performs character recognition processing to recognize characters in various documents and extract attributes within the document. For example, as shown in FIG. 4, items such as title, contractor, contract conclusion date, and user-specified items are extracted as attributes within the document from the character-recognized document. For the title, the title is extracted as a value using words such as contract as a key. For the contractor, the names of contractors A, B, and C are extracted as values. For the contract conclusion date, the contract conclusion date is extracted by pattern matching or the like. For the user-specified items, a character string designated in advance by the user is used as a key and the character string to the right of the key is extracted as a value.

[0037] However, character recognition of a character string that is a key for a document to be obtained may be difficult depending on circumstances such as the background. For example, in documents such as contracts, estimates, and invoices, character strings may be difficult to recognize due to overlapping of a seal impression and characters. In documents such as estimates and invoices, character strings may be difficult to recognize due to halftone dots in tables. In certificates, character strings may be difficult to recognize due to background patterns. In facsimiles, character strings may be difficult to recognize due to low resolution. In recent years, among these processes, image conversion processing using AI (Artificial Intelligence) processing using artificial intelligence that has learned in advance by machine learning is performed as a preprocessing step to remove images other than characters and generate images that are easy to recognize characters, but this processing is very heavy and causes users to wait.

[0038] Therefore, in this embodiment, as a pre-processing for character recognition, the server 14 executes a predetermined image conversion process for the document to be subjected to character recognition for each attribute in the document or for each area in the document that is predetermined according to the type of the document, and performs character recognition on the document that has been subjected to the image conversion process and outputs the result. Note that, below, as an example of executing a predetermined image conversion process for each attribute in a document, an example of switching and executing the image conversion process on a page basis will be described.

[0039] Here, a description will be given of a functional configuration realized by the CPU 14A of the server 14 executing a program stored in the ROM 14B. Fig. 5 is a functional block diagram showing a functional configuration of the server 14 in the information processing system 10 according to this embodiment.

[0040] As shown in FIG. 5, the server 14 according to this embodiment has the functions of a document information acquisition unit 50, a basic preprocessing unit 52, a document type determination unit 54, a preprocessing procedure determination unit 56, a preprocessing unit 58, a character recognition processing unit 60, an attribute extraction unit 62, and a result output unit 64.

[0041] The document information acquisition unit 50 acquires document information from the image forming device 12, the scanner device 13, the mobile terminal 15, or the client terminal 16. In the case of a paper document, the document information acquired is generated by reading the paper document with the image forming device 12 or the scanner device 13.

[0042] The basic preprocessing unit 52 performs basic preprocessing such as detecting the top and bottom of a document, correcting the skew, and identifying the document. For identifying the document, for example, the type of document may be identified by detecting the title or the like by performing simple character recognition on the first page of the document information, or the type of document may be input by the user and the input type of document may be accepted.

[0043] When the basic pre-processing unit 52 performs simplified character recognition to identify a document, the document type determination unit 54 determines the type of the document from the document identified by the basic pre-processing unit 52. When the user inputs the document type, the document information acquisition unit 50 acquires document information and receives the input information, and determines the document type from the received information.

[0044] The preprocessing procedure determination unit 56 acquires information on attributes to be acquired in advance according to the type of document, preprocessing when acquiring attributes in the document, and processing procedures, and determines the procedure of preprocessing. The preprocessing procedure determination unit 56 determines the procedure of preprocessing, for example, using a list that determines attributes such as items to be acquired for each type of document, preprocessing when acquiring attributes in the document, and processing positions. Specifically, as shown in the list in FIG. 6, key words as attributes to be acquired for each document name, processing contents of preprocessing, and processing positions are determined in advance, and corresponding contents are determined according to the type of document. FIG. 6 shows an example of a list that determines key words to be acquired for each type of document, processing contents of preprocessing, and processing positions thereof in advance. In the example in FIG. 6, the title of the contract is the first page in the AI ​​processing that removes the seal impression, the name of the contracting party of the contract is the last page in the AI ​​processing that removes the seal impression, and the contract conclusion date of the contract is the middle page in the dropout color processing. In addition, the title of the quotation is made the first page by AI processing to remove the seal impression, and the quotation source, quotation amount, quotation expiration date, and submission destination are made pages other than the first by dropout color processing. In addition, the title of the invoice and the billing company name are made the first page by AI processing to remove the seal impression, and the invoice amount and billing item are made pages other than the first by dropout color processing.

[0045] The preprocessing unit 58 performs preprocessing on the document information according to the result of the determination by the preprocessing procedure determination unit 56. In this embodiment, the preprocessing determined by the preprocessing procedure determination unit 56 from among a plurality of types of preprocessing is performed. Examples of the plurality of types of preprocessing include a plurality of types of AI processing as an example of a first image conversion processing, and image conversion processing such as dropout color, screen image density processing, and sharpness adjustment as an example of a second image conversion processing. The AI ​​processing is a processing to remove images other than characters by performing image conversion according to the image by artificial intelligence processing using an artificial intelligence model that has been machine-learned in advance. The AI ​​processing is a processing to learn for each target other than characters to be removed. The dropout color is a processing in which the character recognition accuracy is lower than that of the AI ​​processing but the processing speed is faster, and it is a processing to binarize each color and remove an image of a desired color using a predetermined threshold value. The screen image density processing is a processing to adjust the density of an image. The sharpness adjustment is a processing to adjust the degree of emphasis of the contour of an image.

[0046] The character recognition processing unit 60 generates character information by recognizing characters from the document information after the image conversion processing by the preprocessing unit 58. The character recognition processing is performed by a well-known technique.

[0047] The attribute extraction unit 62 extracts attributes such as items within a document based on character information generated by character recognition processing.

[0048] The result output unit 64 outputs the extraction result of the attribute extraction unit 62 to the device that has made the request. For example, it outputs to the image forming device 12, the scanner device 13, the mobile terminal 15, or the client terminal 16.

[0049] Next, a specific process performed by the server 14 of the information processing system 10 according to the present embodiment configured as described above will be described. Fig. 7 is a flowchart showing an example of the flow of the process performed by the server 14 of the information processing system 10 according to the present embodiment. The process in Fig. 7 starts when an instruction to perform character recognition processing is issued by, for example, the image forming device 12, the scanner device 13, the mobile terminal 15, or the client terminal 16.

[0050] In step 100, CPU 14A acquires document information and proceeds to step 102. That is, document information acquisition unit 50 acquires document information from image forming device 12, scanner device 13, mobile terminal 15, or client terminal 16. In the case of a paper document, document information generated by reading the paper document with image forming device 12 or scanner device 13 is acquired.

[0051] In step 102, the CPU 14A performs basic pre-processing on the acquired document information, and the process proceeds to step 104. That is, the basic pre-processing unit 52 performs basic pre-processing such as detecting the top and bottom of the document, correcting the skew, and specifying the document.

[0052] In step 104, CPU 14A determines the type of document and proceeds to step 106. That is, when document type determination unit 54 specifies a document by performing simple character recognition using basic pre-processing unit 52, the type of document is determined from the document specified by basic pre-processing unit 52. When the user inputs the type of document, document information acquisition unit 50 accepts the information input together with acquisition of document information and determines the type of document from the accepted information.

[0053] In step 106, CPU 14A performs a processing procedure determination process, and proceeds to step 108. In the processing procedure determination process, preprocessing procedure determination unit 56 determines a preprocessing procedure based on a list of key words to be acquired, the processing contents of preprocessing, and the processing position of preprocessing, which are determined in advance for each document type. For example, the processing procedure is determined according to the document type based on the list shown in Fig. 6. Specifically, when the document type is a contract, the first page with the title is processed by AI, the last page with the contracting party's name is processed by AI, and the pages with the contract conclusion date are processed by dropout color.

[0054] In step 108, the CPU 14A performs preprocessing for each page and proceeds to step 110. That is, the preprocessing unit 58 performs preprocessing on the document information, focusing on one page, in accordance with the result of the determination by the preprocessing procedure determination unit 56. In this embodiment, the preprocessing determined by the preprocessing procedure determination unit 56 from among a plurality of types of preprocessing is performed. For example, if the document is a contract, the first page with the title and the last page with the contracting party's name are preprocessed by AI processing to remove the seal impression, and the pages between the first and last pages with the contract conclusion date and the like are preprocessed by dropout color processing.

[0055] In step 110, the CPU 14A performs character recognition processing on the preprocessed page, and proceeds to step 112. That is, the character recognition processing unit 60 recognizes characters from the document information after the preprocessing by the preprocessing unit 58, and generates character information.

[0056] In step 112, the CPU 14A extracts attributes from the character information generated by the character recognition process, and proceeds to step 114. That is, the attribute extraction unit 62 extracts attributes of items and the like in the document based on the character information generated by the character recognition process.

[0057] In step 114, the CPU 14A judges whether or not the attribute acquisition is completed. This judges whether or not there are any remaining pages to be preprocessed and character recognition processed. If the judgement is negative, the process proceeds to step 108, where the above-mentioned process is repeated for the remaining pages, and if the judgement is positive, the process proceeds to step 116.

[0058] In step 116, CPU 14A outputs the result of the attribute extraction and ends the series of processes. That is, result output unit 64 outputs the extraction result of attribute extraction unit 62 to the requesting device. For example, it outputs to image forming device 12, scanner device 13, mobile terminal 15, or client terminal 16.

[0059] In this manner, in this embodiment, for example, for attributes such as the title of a contract or the name of a contractor, pages where the seal impression is likely to overlap with the characters undergo image conversion processing using AI processing as preprocessing. On the other hand, for other pages, image conversion processing using dropout color processing, which has a lower processing load and faster processing speed than AI processing, is performed as preprocessing. In other words, by performing preprocessing image conversion processing that is predetermined for each attribute in a document, both processing speed and character recognition accuracy are achieved compared to the case where a single image conversion processing is performed as preprocessing.

[0060] In the above embodiment, an example has been described in which the preprocessing is performed in order without changing the pages, but the order of the pages to be processed may be changed before performing the preprocessing.

[0061] Here, a modified example will be described in which the page order is changed and preprocessing is performed. In this case, a list with a predetermined processing order is used as shown in FIG. 8 for the list in FIG. 6. FIG. 8 shows an example of a list in which the key words to be acquired for each type of document, the processing contents of the preprocessing, and the processing order are predetermined. In the example in FIG. 8, the title of the contract is processed by AI processing to remove the seal imprint, the name of the contracting party of the contract is processed by AI processing to remove the seal imprint, the contract conclusion date of the contract is processed by dropout color processing, and the others (individual setting attributes) are not preprocessed. The processing order is the first, the last, the second page from the first, the second page from the last, the third page from the first, etc. The title of the quotation is processed by AI processing to remove the seal imprint, the quotation originator, the quotation amount, the quotation expiration date, and the submission destination of the quotation are processed by dropout color processing, and the others (individual setting attributes) are not preprocessed. The processing order is performed sequentially from the first page. In addition, the invoice title and billing company name are processed with AI to remove the seal impression, the invoice amount, billing item, payee, and payment due date are processed with dropout color, the invoice destination is processed with AI, the invoice table information extraction is processed with AI to remove halftone dots, and the other invoice attributes (individually set attributes) are not preprocessed. And the processing order is first, last, second page from the first, second page from the last, third page from the first, etc.

[0062] For example, if the document is a contract, the same preprocessing is executed all at once by switching the processing order as shown in Fig. 9. In the example of Fig. 9, preprocessing is executed in the order of the first page of Article 1, where the seal impression may overlap with the text, the last page of Article 10, the page of Article 2, the page of Article 9, the page of Article 3, the page of Article 8, the page of Article 4, the page of Article 7, the page of Article 5, and the page of Article 6.

[0063] Next, a specific process performed by the server 14 of the information processing system 10 when preprocessing is performed after changing the page order of the contract will be described. Fig. 10 is a flowchart showing an example of the process flow when preprocessing is performed after changing the page order of the contract in the server 14 of the information processing system 10 according to this embodiment. The process in Fig. 10 starts when an instruction to perform character recognition processing is issued by, for example, the image forming device 12, the scanner device 13, the mobile terminal 15, or the client terminal 16.

[0064] In step 200, the CPU 14A acquires document information of the contract, and proceeds to step 202. That is, the document information acquisition unit 50 acquires document information of the contract from the image forming device 12, the scanner device 13, the mobile terminal 15, or the client terminal 16. In the case of a paper document, the document information of the contract generated by reading the paper document of the contract by the image forming device 12 or the scanner device 13 is acquired.

[0065] In step 202, the CPU 14A performs basic pre-processing on the acquired document information of the contract, and proceeds to step 204. That is, the basic pre-processing unit 52 performs basic pre-processing such as detecting the top and bottom of the document, correcting the skew, and specifying the document.

[0066] In step 204, CPU 14A determines that the document type is a contract, and proceeds to step 206. That is, when document type determination unit 54 specifies the document by performing simplified character recognition using basic pre-processing unit 52, it determines that the document type is a contract from the document specified by basic pre-processing unit 52. When the user is prompted to input the document type, document information acquisition unit 50 accepts the information input together with the acquisition of document information, and determines that the document type is a contract from the accepted information.

[0067] In step 206, CPU 14A performs a processing procedure determination process and proceeds to step 208. In the processing procedure determination process, preprocessing procedure determination unit 56 determines a preprocessing procedure based on a list of keyword to be acquired, processing contents of preprocessing, and processing order of preprocessing, which are determined in advance for each document type. For example, the processing procedure is determined according to the document type based on the list shown in FIG. 8. Specifically, when the document type is a contract, a page having a title and a contracting party name is subjected to AI processing, a page having a contract conclusion date is subjected to dropout color processing, and a page having other (individual setting attributes) is subjected to no preprocessing, and the processing order is set to first, last, second page from the first, second page from the last, third page from the first, and so on.

[0068] In step 208, the CPU 14A performs AI processing as preprocessing, and proceeds to step 210. That is, the preprocessing unit 58 performs AI processing for each page according to the determination result of the preprocessing procedure determination unit 56. Here, the AI ​​processing is performed on the first page of the title and the last page of the contractor name.

[0069] In step 210, the CPU 14A performs character recognition processing on the preprocessed pages, and proceeds to step 212. That is, the character recognition processing unit 60 recognizes characters from the document information for the first and last pages after preprocessing by the preprocessing unit 58, and generates character information.

[0070] In step 212, the CPU 14A extracts attributes from the character information generated by the character recognition process, and proceeds to step 214. That is, the attribute extraction unit 62 extracts the title and the contractor name in that order as attributes of items, etc. in the document, based on the character information generated by the character recognition process.

[0071] In step 214, the CPU 14A judges whether or not the title and the contractor name have been acquired. This judgement is made by judging whether or not the final contractor name has been extracted after the title has been extracted from the first page. If only the title has been extracted and the contractor name has not been extracted, the judgement is negative and the process returns to step 208 to repeat the above-mentioned process for the next page, and if the judgement is positive, the process proceeds to step 216.

[0072] In step 216, CPU 14A performs color dropout processing as preprocessing, and proceeds to step 218. That is, preprocessing unit 58 performs color dropout processing for each page in accordance with the determination result of preprocessing procedure determination unit 56. Here, color dropout processing is performed on the second page from the first, the second page from the last, the third page from the first, and so on.

[0073] In step 218, CPU 14A performs character recognition processing on the preprocessed page, and proceeds to step 220. That is, character recognition processing unit 60 recognizes characters from the document information preprocessed by preprocessing unit 58, and generates character information. Here, character recognition processing is performed on document information that has been subjected to dropout color processing, and character information is generated.

[0074] In step 220, the CPU 14A extracts attributes from the character information generated by the character recognition process, and proceeds to step 222. That is, the attribute extraction unit 62 extracts the contract conclusion date, etc., as attributes of items, etc. in the document, based on the character information generated by the character recognition process.

[0075] In step 222, the CPU 14A determines whether or not the attribute acquisition has been completed. If the determination is negative, the process returns to step 216 and the above-mentioned processing is repeated. If the determination is positive, the process proceeds to step 224.

[0076] In step 224, CPU 14A outputs the result of the attribute extraction and ends the series of processes. That is, result output unit 64 outputs the extraction result of attribute extraction unit 62 to the requesting device. For example, it outputs to image forming device 12, scanner device 13, mobile terminal 15, or client terminal 16.

[0077] In the above embodiment, an example has been described in which a predetermined image conversion process for each attribute in a document is performed on a page-by-page basis as pre-processing, but this is not limited to page-by-page basis. For example, if the location of an attribute such as the title of a contract on a page is determined, the image conversion process of the pre-processing may be switched not on a page-by-page basis but on a region-by-region basis within a page. For example, if the region of the title of an invoice is located in the upper region of a page, the predetermined region on the upper side of the first page may be subjected to AI processing, and the other regions of the first page may be subjected to other image conversion processes (e.g., dropout color processing, etc.) other than AI processing.

[0078] In the above embodiment, the AI ​​processing is an example of the first image conversion processing, and the dropout color processing is an example of the second image conversion processing, but the present invention is not limited to this, and the first image conversion processing and the second image conversion processing may be determined according to the character recognition accuracy and the processing speed. In addition, when the character recognition accuracy and the processing speed are different even within the AI ​​processing, the first image conversion processing and the second image conversion processing may be determined within the AI ​​processing. In addition, an image conversion processing that has a slower processing speed than the AI ​​processing but a higher character recognition accuracy may be set as the first image conversion processing, and another AI processing may be set as the second image conversion processing.

[0079] In addition, in the above embodiment, the CPU has been described as an example of a processor, but the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPUs, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic devices, etc.).

[0080] In addition, the operations of the processor in the above embodiments may be performed not only by one processor, but also by multiple processors located at physically separate locations working together. The order of the operations of the processor is not limited to the order described in the above embodiments, and may be changed as appropriate.

[0081] The processing performed by the server 14 according to the above embodiment may be processing performed by software, processing performed by hardware, or a combination of both. The processing performed by the server 14 may be stored as a program in a storage medium and distributed.

[0082] Furthermore, the present invention is not limited to the above, and it goes without saying that various modifications can be made without departing from the spirit and scope of the present invention. [Explanation of symbols]

[0083] 10. Information Processing Systems 12 Image forming device 13 Scanner device 14 Server 14A CPU 15 Mobile devices 16 Client Terminals 50 Document Information Acquisition Unit 52 Basic preprocessing section 54 Document type determination section 56 Pre-processing procedure determination unit 58 Pretreatment section 60 Character recognition processing section 62 Attribute extraction part 64 Result output section

Claims

1. a processor, the processor comprising: As a pre-processing step for character recognition, a predetermined image conversion process is performed on the document to be subjected to character recognition for each attribute in the document or for each area in the document that is predetermined according to the type of the document; An information processing apparatus that performs character recognition on the document that has been subjected to the image conversion processing and outputs the results, and executes the image conversion processing by rearranging the page order so that processing is performed for each image conversion processing.

2. The information processing device according to claim 1 , wherein the image conversion process includes a first image conversion process that removes non-characters, and a second image conversion process that has a lower character recognition accuracy than the first image conversion process but has a faster processing speed for removing non-characters.

3. The information processing apparatus according to claim 2 , wherein the first image conversion process is an image conversion process using artificial intelligence that has been trained in advance by machine learning.

4. The information processing apparatus according to claim 3 , wherein the first image conversion process includes a plurality of different types of image conversion processes each of which is different for a target other than characters to be removed.

5. 5. The information processing apparatus according to claim 1, wherein the processor executes the image conversion process by rearranging the pages in a predetermined page order for each type of the document.

6. An information processing device according to any one of claims 1 to 5; a requesting device that transmits a read result of the document to the information processing device to request character recognition; An information processing system comprising:

7. On the computer, As a pre-processing step for character recognition, a predetermined image conversion process is performed on the document to be subjected to character recognition for each attribute in the document or for each area in the document that is predetermined according to the type of the document; An information processing program for executing a process of performing character recognition on the document that has been subjected to the image conversion process and outputting the results, and for executing the image conversion process by rearranging the page order so that processing is performed for each image conversion process.

Images