Image processing apparatus, image processing method, and image processing program

Abstract

To provide an image processing device, an image processing method and an image processing program capable of improving examination efficiency.SOLUTION: An image processing device (an information processing device 100) includes: an application information acquisition part 11 for acquiring application image data, the image data of application documents of at least one attribute out of an application form containing application information of an applicant out of application documents submitted by the applicant and a certificate of the applicant; and an image processing part 30 determining the attribute of the application documents represented by the application image data, performing image processing of the application image data according to the attribute, and generating examination image data for use in examination of a content of the application information.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to an image processing apparatus, an image processing method, and an image processing program.

Background Art

[0002] Conventionally, for example, when applying to a local government, the local government accepts application documents from an applicant and examines whether there are any deficiencies in the contents of the application documents and the like. The local government receives, for example, a plurality of types of documents such as an application form together with a copy of an identity document from an applicant and a certificate for confirming the application content from one applicant, and examines whether there are any deficiencies in the application content by comparing these multiple documents. Similar processing is also performed in applications for services to private companies. At this time, since these multiple types of application documents are in paper media, the efficiency of the application information confirmation work and the input work of the confirmation results may be deteriorated. Therefore, for example, as a preprocess of the examination, the application documents may be read by a scanner or the like, and the entered characters may be automatically converted into data and stored in a storage device (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the application documents to be examined include an application form, an identity document, etc., and the document format and the printing format on the paper may differ depending on the document. For this reason, even if the application documents are simply converted into image data, it takes time to display the necessary application document data for each confirmation item, and the examination efficiency may not be improved. Therefore, an object of the present disclosure is to provide an image processing apparatus, an image processing method, and an image processing program capable of improving the examination efficiency.

Means for Solving the Problem

[0005] To achieve the above object, an image processing apparatus according to an aspect of the present disclosure includes an image acquisition unit that acquires application image data, which is image data of application documents including at least one of an application form including applicant information of the applicant and the applicant's certificate documents among the application documents submitted by the applicant, and an image processing unit that determines which of the application form or the certificate documents the attribute of the application documents indicated by the application image data is, performs image processing on the application image data according to the attribute, and generates review image data used for reviewing the content of the application information. The application image data includes certificate copy image data, which is an image of a copy of the certificate documents and in which part of the certificate documents is printed. The paper surface of the paper medium The image area of the certificate copy image data includes a certificate image area showing the whole of the certificate documents printed thereon and a background image area showing a margin where the certificate documents are not arranged. The image processing unit has a learned model capable of classifying the attribute of the application documents, determines which of the application form or the certificate documents the attribute is based on a classification result obtained by inputting the application image data into the learned model, and further, as the image processing, discriminates the certificate image area and the background image area with respect to the image area of the certificate copy image data in the application image data, cuts out the certificate image area excluding the background image area from the image area of the certificate copy image data, and can execute a process of generating certificate image data, which is new image data including the certificate image area, as the review image data, and determines whether to execute the image processing based on the determination result of the attribute. Within the paper surface The whole of the certificate documents printed thereon Within the said paper surface The image area of the certificate copy image data includes a certificate image area showing the whole of the certificate documents printed thereon and a background image area showing a margin where the certificate documents are not arranged. The image processing unit has a learned model capable of classifying the attribute of the application documents, determines which of the application form or the certificate documents the attribute is based on a classification result obtained by inputting the application image data into the learned model, and further, as the image processing, discriminates the certificate image area and the background image area with respect to the image area of the certificate copy image data in the application image data, cuts out the certificate image area excluding the background image area from the image area of the certificate copy image data, and can execute a process of generating certificate image data, which is new image data including the certificate image area, as the review image data, and determines whether to execute the image processing based on the determination result of the attribute. Indicating within the said paper surface The image area of the certificate copy image data includes a certificate image area showing the whole of the certificate documents printed thereon and a background image area showing a margin where the certificate documents are not arranged. The image processing unit has a learned model capable of classifying the attribute of the application documents, determines which of the application form or the certificate documents the attribute is based on a classification result obtained by inputting the application image data into the learned model, and further, as the image processing, discriminates the certificate image area and the background image area with respect to the image area of the certificate copy image data in the application image data, cuts out the certificate image area excluding the background image area from the image area of the certificate copy image data, and can execute a process of generating certificate image data, which is new image data including the certificate image area, as the review image data, and determines whether to execute the image processing based on the determination result of the attribute.

[0006] Also, in order to achieve the above object, an image processing method according to one aspect of the present disclosure is an image processing method executed by a computer, which acquires application image data that is image data of application documents including at least one of an application form containing applicant's application information and the applicant's certificate documents among the application documents submitted by the applicant, determines whether the attribute of the application documents indicated by the application image data is the application form or the certificate documents, performs image processing on the application image data according to the attribute to generate review image data used for reviewing the content of the application information, and the application image data includes certificate copy image data that is an image data of a copy of the certificate documents and in which a part of the certificate documents is printed. The paper surface of the paper medium The image area of the certificate copy image data includes a certificate image area showing the whole of the certificate documents printed on the Within the paper surface and a background image area showing a blank space where the certificate documents are not arranged. As the image processing, based on a classification result obtained by inputting the application image data into a learned model capable of classifying the attribute of the application documents, it is determined whether the attribute is the application form or the certificate documents. Further, as the image processing, the certificate image area and the background image area are discriminated from the image area of the certificate copy image data in the application image data, the certificate image area obtained by removing the background image area from the image area of the certificate copy image data is cut out, and a process of generating certificate image data, which is new image data including the certificate image area, as the review image data can be executed, and it is characterized in that it is determined whether to execute the image processing based on the determination result of the attribute. Within the said paper surface Indicating within the said paper surface

[0007] ​​Also, in order to achieve the above object, an image processing program according to one aspect of the present disclosure acquires application image data that is image data of at least one of an application form including applicant's application information and the applicant's certificate documents among the application documents submitted by the applicant, determines whether the attribute of the application documents indicated by the application image data is the application form or the certificate documents, performs image processing on the application image data according to the attribute, and generates review image data used for reviewing the content of the application information. The application image data includes certificate copy image data that is an image of a copy of the certificate documents and in which a part of the certificate documents is printed, and the image area of the certificate copy image data includes a certificate image area showing the entire certificate documents printed on the [ The paper surface of the paper medium and a background image area showing a blank space where the certificate documents are not arranged. As the image processing, based on a classification result obtained by inputting the application image data into a learned model capable of classifying the attribute of the application documents, it is determined whether the attribute is the application form or the certificate documents. Further, as the image processing, the certificate image area and the background image area are discriminated from the image area of the certificate copy image data in the application image data, and the certificate image area obtained by removing the background image area from the image area of the certificate copy image data is cut out, and certificate image data that is new image data including the certificate image area is generated as the review image data. It is possible to execute a process, and based on the determination result of the attribute, it is determined whether or not to execute the image processing, and the computer is caused to execute the process. Within the paper surface The above is characterized by Within the said paper surface

Effects of the Invention

Brief Description of the Drawings

[0008]

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. However, the embodiments described below are merely examples and are not intended to exclude various modifications and applications of technologies not explicitly described below. The present disclosure can be implemented with various modifications (for example, combining each embodiment) without departing from its gist. Also, in the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

[0011] In addition, the following image processing apparatus is realized by the cooperation of hardware, which is an information processing terminal device such as a general-purpose computer, and software, which is a computer program. The computer program is readable by one or more processors included in the image processing apparatus, and when installed on a hard disk, the processor functions as each part of the image processing apparatus.

[0012] Hereinafter, the image processing apparatus according to the present embodiment, and an image processing method and an image processing program executed by the image processing apparatus will be described with reference to FIGS. 1 to 12.

[0013] 1. Overall configuration of the information processing system As shown in FIG. 1, the information processing system 1 is used as an application information processing system that acquires application information applied by an applicant and conducts a review of the application content. In FIG. 1, as an example of the information processing system 1, an application information processing system that receives an application procedure from an applicant as an internal system of a local government and obtains the result of the review by a reviewer is illustrated. Note that this information processing system 1 may be a system of a company or the like commissioned by a local government, or may be an internal system of a company that receives an application procedure (application procedure) from an applicant for a service. The information processing system 1 is connected to a system of a printing department inside or outside the local government and can create printed materials such as a notice according to the output from the information processing system 1 by a printing device. Also, the information processing system 1 may be connected to a system of a bank and perform a transfer or the like according to the review.

[0014] FIG. 2 is a schematic functional block diagram showing an example of the functional configuration of the information processing system 1. As shown in FIG. 2, the information processing system 1, which is an application information processing system, includes an information processing apparatus 100. The information processing apparatus 100 functions as an application information processing apparatus that acquires application information applied by an applicant and performs a review of the application content. In the present embodiment, the information processing apparatus 100 includes an image processing unit 30. Thereby, it functions as an image processing apparatus that converts an application document on paper into image data, performs image processing, and generates image data suitable for review. That is, the information processing apparatus 100 corresponds to the image processing apparatus according to the present embodiment.

[0015] When receiving an application procedure from an applicant, a local government or a company, which is a review authority, can receive an application document on paper from the applicant by methods such as mail or visiting the review authority, or receive an application document from the applicant via the Internet. In the present embodiment, as an example, the system configuration and the information processing apparatus 100 when the applicant submits an application document on paper by methods such as mail or visiting the local government will be described.

[0016] 2. Functional Configuration of Information Processing Apparatus As shown in FIG. 2, the information processing apparatus 100 (an example of an image processing apparatus) is, for example, a computer. Each functional block shown in FIG. 2 is realized by a processor 101 of an arithmetic unit 10, which will be described later in FIG. 9, executing a necessary program and controlling the operations of each part of the information processing apparatus 100.

[0017] The information processing apparatus 100, which is an image processing apparatus according to this embodiment, includes an application information acquisition unit 11, a storage unit 12, a review unit 13, a display data output unit 14, a data output unit 15, an image processing unit 30, and a review image storage unit 31. Further, the information processing apparatus 100 is connected to a reading device 51 such as a scanner, which is an external device, an input device 52 such as a keyboard and a mouse, a display device 53 such as a liquid crystal display, and an OCR review server 54. The OCR review server 54 is a server device that reviews the application content based on the result of OCR (Optical Character Recognition) processing on the image data generated by the information processing apparatus 100 (particularly, the image processing unit 30). Note that the reading device 51, the input device 52, and the display device 53 may be provided as a part of the information processing apparatus 100. Also, the function of the OCR review server 54 may be realized as a functional block of the information processing apparatus 100.

[0018] Also, the OCR engine in the OCR review server 54 may be an AI-OCR engine to which AI technology is applied. The AI-OCR engine can improve the accuracy of character recognition and the accuracy of layout analysis (specification of the recognition range) of image data by deep learning. When using the AI-OCR engine, the OCR review server 54 is preferably a cloud server provided in an open network environment (cloud environment). Thereby, the AI-OCR engine can learn from a large amount of data, and the accuracy of the OCR processing result can be improved. Hereinafter, each functional block will be described.

[0019] [Application Information Acquisition Unit] The application information acquisition unit (an example of an image acquisition unit) 11 is an interface that acquires application image data obtained by digitizing application documents from an applicant. As shown in FIG. 2, the information processing apparatus 100 is connected to a reading device 51 such as a scanner that reads the application documents in which the application information is entered, digitizes the application documents, and creates the application image data.

[0020] The application image data is the image data of the application documents (such as application forms and certificates) on paper submitted by the applicant, and contains the applicant's application information. The application documents submitted by the applicant are classified into multiple types of attributes. Examples of the attributes of the application documents in this embodiment include, for example, an application form containing the applicant's information (with the application information filled in) and the certificates attached to the application form. Here, the applicant may include not only the applicant himself / herself but also the applicant's family members, etc. The certificates are, for example, copies of the applicant's (the applicant and the applicant's family members, etc.) identity verification documents and certificates for verifying the application content. Note that the certificates are treated as attached documents to the application form. In addition to the above-mentioned certificates, the attached documents may include various forms such as invoices and order forms in some cases.

[0021] When the application image data is generated by the application documents being read by a reading device 51 such as a scanner, the application image data contains the applicant's application information as image information. Thus, in this embodiment, the application information acquisition unit 11 acquires application image data that is the image data of application documents of at least one of the attributes of the application form containing the applicant's application information and the applicant's certificates among the application documents submitted by the applicant.

[0022] FIG. 3 and FIG. 4 are diagrams showing an example of the application image data acquired by the application information acquisition unit 11. Specifically, FIG. 3 is an example of application form reading image data that is the image data of the application form read by the reading device 51 among the application image data. The application form reading image data 71 is the image data obtained by reading the application form itself with the reading device 51 (such as a scanner), and is formed by an application form image area 710 that shows the content of the application form as image information.

[0023] In this example, the application form image area 710 includes an application item area containing image information indicating the types of various information regarding the applicant and the application content, and an application content area containing image information indicating the content of the various information. For example, in FIG. 3, the application item area 711, which is an example of the application item area, includes image information of the character string "Name", which indicates the type of information regarding the applicant. Also, the application content area 712, which is an example of the application content area, includes image information of the character string "Taro, Japan", which indicates the content of the information regarding the applicant and corresponds to the information content of the application item area 711. In this example, the application item area is associated with one or more application content areas.

[0024] Also, FIG. 4 is an example of certificate reading image data, which is image data of certificates read by the reading device 51 among the application image data. The certificate reading image data 72 is image data obtained by reading a copy (printed matter) of the certificates submitted by the applicant in paper form with the reading device 51 (such as a scanner). Therefore, the certificate reading image data 72 is formed by including a certificate image area 720 that shows the content of the certificates as image information and a background image area 721. The background image area 721 is an image area that shows the blank portion where the certificates are not placed during the printing of the certificates by a predetermined printing device (such as a printer or a copier).

[0025] Also, the certificates submitted by the applicant as attached documents to the application form are not limited to one type and may be multiple types. In this case, the applicant may print multiple types of certificates on a single sheet of printing paper using a printing device. That is, one certificate reading image data 72 may include multiple certificate image areas 720. In this example, an example in which the certificate reading image data 72 includes multiple certificate image areas 720 (certificate image areas 720a and 720b) is illustrated. In FIG. 4, the certificate image area 720a is a certificate image area that shows the content of the insurance certificate as image information, and the certificate image area 720b is a certificate image area that shows the content of the driver's license as image information.

[0026] In this example, the certificate image area 720 includes a type area containing image information indicating the type of the certificate, a certificate item area containing image information indicating the types of various information regarding the applicant and the certificate content, and a certificate content area containing image information indicating the content of the various information. For example, in FIG. 4, the type area 722, which is an example of the type area, contains image information of character strings such as "insurance certificate" and "driver's license" indicating the type of the certificate. Also, the certificate item area 723, which is an example of the certificate item area, contains image information of the character string "name" indicating the type of information regarding the applicant. Further, the certificate content area 724, which is an example of the certificate content area, contains image information of the character string "Taro Japan" indicating the content of the information regarding the applicant and corresponding to the certificate item area 723 (in this example, the information indicating the "name" of the applicant). In this example, the certificate item area is associated with one or more application content areas.

[0027] In actuality, the certificate image area 720 contains a wide variety of certificate item areas and corresponding certificate content areas. In this example, for ease of understanding, only the certificate item area 723 and the certificate content area 724, which are a part of many lighting item areas and certificate content areas, are illustrated, and the illustration of other certificate item areas and certificate content areas is omitted.

[0028] The above has described the application image data in this embodiment with specific examples. In the following description, the application item area (for example, the application item area 711) in the application form image area 710 and the certificate item area (for example, the certificate item area 723) in the certificate image area 720 may be collectively referred to simply as the "item area". Also, the application content area (for example, the application content area 712) in the application form image area 710 and the certificate content area (for example, the certificate content area 724) in the certificate image area 720 may be collectively referred to simply as the "content area".

[0029] Returning to FIG. 2, when the application image data is generated in the reading device 51, the generated application image data is output to the application information acquisition unit 11. When the application information acquisition unit 11 acquires the application image data input from the reading device 51, it outputs the data to the storage unit 12. As a result, the application image data (for example, the application form reading image data 71 and the certificate reading image data 72) is stored in the storage unit 12.

[0030] Note that the application image data acquired by the application information acquisition unit 11 is not limited to that output from the reading device 51. For example, the reading device 51 may output the application image data to the storage unit 12, and the application information acquisition unit 11 may acquire the application image data stored in the storage unit 12. Also, the application image data may be output from, for example, a computer device (not shown) for acquiring image data connected to the information processing device 100 and acquired by the application information acquisition unit 11. The computer device for image acquisition only needs to be connected to a reading device such as a scanner that reads application documents and creates and outputs application image data in the same manner as the reading device 51. When the computer device for image acquisition acquires the application image data output from the reading device, it outputs the acquired application image data to the information processing device 100. As a result, the application information acquisition unit 11 can acquire the application image data from the computer device for image acquisition. Note that the application image data output by the computer device for image acquisition may be stored in the storage unit 12, and the application information acquisition unit 11 may read the application image data from the storage unit 12. In this way, the information processing device 100 (particularly the application information acquisition unit 11) may be configured not to be connected to the reading device 51 (a configuration that is not integrated with the reading device 51 but is independent).

[0031] [Storage Unit] The storage unit 12 stores the application image data acquired by the application information acquisition unit 11 in association with the applicant information indicating the applicant. In addition, the storage unit 12 stores information input via an input device 52 such as a keyboard by a person in charge of examination who examines application documents. For example, as will be described later, the examination unit 13 displays an examination screen 61 on the display device 53 on which examination items such as "Please confirm and enter the name in the identity document" and "Please confirm whether the name in the identity document matches the application form" are displayed, and allows the person in charge of examination to input examination results by means of character input, selection of a checklist, etc. The storage unit 12 stores information indicating the examination results input by the person in charge of examination via the input device 52. In addition, the storage unit 12 stores attribute information (for example, information indicating an applicant, etc.) corresponding to each application data input via the input device 52. In addition, the storage unit 12 stores various information referred to in the image processing in the image processing unit 30. The storage unit 12 also stores the application image data and applicant information indicating the applicant in association with the examination image data generated by the image processing unit 30. Thereby, in the information processing apparatus 100, it is possible to specify which applicant's examination case the examination image data is related to, and from which application image data the examination image data is generated.

[0032] [Examination Department] The examination unit 13 has a function of generating an examination screen on which examination items for examining the content of the application image data, that is, the content of the application documents are displayed, displaying the examination screen and the examination image data based on the application image data side by side on the display device 53, and acquiring the examination results and storing them in the storage unit 12. Since the confirmation of the content of the application documents is performed by visual inspection of the examination screen and the examination image data by the person in charge of examination, the examination by this function is also called visual examination. More specifically, the examination unit 13 generates electronic data (examination screen data) of the examination screen 61, and acquires input data for acquiring information (for example, examination results input by the person in charge of examination) input from the input device 52 by the person in charge of examination during the examination.

[0033] In addition, the examination unit 13 synthesizes the electronic data of the examination screen and the input data corresponding to the input examination results, and causes the display device 53 to display them. For example, when a radio button is checked using a mouse, the examination unit 13 generates display screen data of the examination screen in a state where the corresponding radio button is selected and displayed on the examination screen according to the position of the cursor. Similarly, the examination unit 13 may generate display screen data of the examination screen in a state where a check mark is displayed in a check box or characters and numbers are input and displayed in a text box. The examination unit 13 acquires input data (information input from the input device 52) corresponding to the examination results for each examination item. The examination unit 13 acquires input data corresponding to the examination results for all examination items for one examination image data, and generates examination data for one examination image data (that is, one applicant). The examination unit 13 associates the examination data with each application image data (each applicant) and outputs it to the storage unit 12 for storage.

[0034] [Image processing unit] The image processing unit 30 performs image processing on the application image data and generates examination image data used for examining the content of the application information (application image data) in the examination unit 13. For example, the image processing unit 30 generates, as the examination image data, image data (examination image data for visual examination) that the examination unit 13 displays on the display device 53 side by side with the examination screen from the application image data. In addition, the image processing unit 30 can generate image data (examination image data for automatic examination described later) output to the OCR examination server 54 from the application image data. The image processing unit 30 outputs the generated examination image data to the examination image storage unit 31 for storage. The configuration of the image processing unit 30 and the details of the examination image data will be described later. Note that the generation of the review image data in the image processing unit 30 is not limited to the generation of new image data based on the application image data as described above. For example, in generating the review image data, information corresponding to the application documents indicated by the application image data may be added to the application image data and stored in the review image storage unit 31. That is, the application image data with the added information may be treated as the review image data.

[0035] [Review Image Storage Unit] The review image storage unit 31 stores the review image data generated by the image processing unit 30. Specifically, the review image storage unit 31 is a predetermined storage area that stores the review image data in a file format. When the image processing unit 30 generates the review image data, it outputs the review image data to the review image storage unit 31 for storage. Also at this time, the image processing unit 30 generates storage destination data indicating the storage destination (e.g., directory, folder) of the review image data in the review image storage unit 31, and stores it in the storage unit 12 in association with at least one of the application image data and the applicant information acquired by the application information acquisition unit 11. The storage destination data may be, for example, a file path or a URL (Uniform Resource Locator). Thereby, for example, when generating the review screen, the review unit 13 can acquire the review image data corresponding to the application image data and display the review screen and the review image data side by side on the display device 53.

[0036] [Display Data Output Unit] The display data output unit 14 outputs the display data generated by the review unit 13 and the review image data generated by the image processing unit 30 to the display device 53.

[0037] [Data Output Unit] The data output unit 15 outputs, in response to an instruction from the input device 52, the examination data corresponding to the application image data stored in the storage unit 12 to the outside of the information processing apparatus 100. The examination data includes, for example, a list of the content of the procedure for which the applicant has applied (for example, the procedure for receiving benefits, etc.) and the examination results (for example, approval or rejection) of the application content of each applicant. The data output unit 15 may output the examination data for one or a plurality of designated applicants, or may output all the examination data stored in the storage unit 12 in a batch. Also, in response to the input of the examination image data from the image processing unit 30, the data output unit 15 outputs the input examination image to the outside of the information processing apparatus 100. Note that the data output unit 15 may read the examination image data stored in the examination image storage unit 31 and output it to the outside of the information processing apparatus 100.

[0038] The data output unit 15 functions as a communication interface or an external interface, and outputs the examination data to a computer connected via a network 9 such as the Internet or a LAN (Local Area Network), or other external storage devices or external storage media. The data output unit 15 can output the examination data to a predetermined computer within the local government connected via, for example, a LAN. Also, the data output unit 15 can output the examination data to a predetermined external computer connected via the Internet. For this reason, the information processing system 1 may be operated within the local government, or may be operated by an external company that has contracted the examination from the local government. When the information processing system 1 is operated by an external company, the company can receive applications, create application data, conduct examinations, create examination data, etc., and transmit the examination data, or both the application data and the examination data, to the local government via the Internet.

[0039] In addition, the data output unit 15 can output the image data for review to an OCR review server 54, which is, for example, a server device connected via the Internet. As a result, in the OCR review server 54, the character information of the image information (the content of the application form and certificates) of the image data for review can be read using OCR technology, and the content of the application documents (application forms, certificates, etc.) can be reviewed without the need for visual inspection by the reviewer, that is, automatic review can be performed. That is, the information processing device 100 can perform two types of reviews, namely, the above-mentioned visual review and automatic review, as the review of the content of the application documents.

[0040] Note that in this embodiment, the connection between the information processing device 100 and the OCR review server 54 is not essential. The information processing device 100 may or may not be connected to the OCR review server 54. That is, the information processing device 100 may be configured to be able to perform only visual review without performing automatic review. Therefore, the image processing unit 30 only needs to generate at least the image data for review for visual review. In addition, when the information processing device 100 and the OCR review server 54 are connected, all reviews related to the application content from the applicant may be automatic reviews by the OCR review server 54. Also, a part of the review related to the application content may be an automatic review by the OCR review server 54, and the remaining review may be a visual review by the function of the review unit 13. Further, when the information processing device 100 is capable of performing both visual review and automatic review, for example, review type data indicating whether to perform visual review or automatic review may be stored in the storage unit 12, and it may be configured to be able to switch between visual review and automatic review based on the review type data.

[0041] (1.1) Image Processing in the Image Processing Unit Next, in the information processing device 100 which is an image processing device according to this embodiment, the details of the image processing executed by the image processing unit 30 and the functional configuration for executing the image processing will be described in detail with reference to FIGS. 2 to 8. As shown in FIG. 2, the image processing unit 30 includes a document determination unit 301 that determines the attributes of the application documents indicated by the application image data, a document splitting unit 302 that splits the application image data according to the attributes of the application documents indicated by the application image data, a region extraction unit 303 that extracts a region including the image information to be reviewed from the image information included in the application image data, and a region splitting unit 304 that splits the extracted region from the application image data or the image data generated from the application image data. Although details will be described later, in the information processing apparatus 100, the region splitting unit 304 is not an essential component, and the image processing unit 30 may not have the region splitting unit 304.

[0042] (Document determination unit) The document determination unit 301 reads the application image data from the storage unit 12 and performs image recognition on the read application image data to determine the attributes of the application documents indicated by the application image data. Here, an example of the image recognition process executed by the document determination unit 301 will be described. The document determination unit 301 performs, for example, an image recognition process using the application image data and the image data of the template (template) of the application document (determination image data). As the determination image data, the image data of the templates of a plurality of types of application documents (for example, application forms, insurance certificates, driver's licenses, etc.) that are assumed to be submitted as application documents are used. The document determination unit 301 compares the application image data with the determination image data and derives a template similar to the application document indicated by the application image data. Thereby, the attributes of the application documents indicated by the application image data can be determined.

[0043] In the image recognition process, when the document determination unit 301 derives the template of the application form as a template similar to the application document indicated by the application image data (for example, the application form reading image data 71), it determines that the attribute of the application document is "application form". Also, when the document determination unit 301 derives the template of the driver's license or insurance certificate as a template similar to the application document indicated by the application image data (for example, the certificate reading image data 72), it determines that the attribute of the application document is "certificate documents". Even when the application image data includes images of a plurality of certificate documents as application documents, the document determination unit 301 can determine the attribute ("certificate documents") by image recognition processing.

[0044] The determination image data may include template image data of attached documents other than certificate documents (such as invoices and order forms). Thereby, the document determination unit 301 can also determine the attribute of the application document when the application document indicated by the application image data is an attached document other than certificate documents. Note that the image recognition processing executed by the document determination unit 301 as the determination of the attribute of the application document is not limited to the above processing. The document determination unit 301 can use known image recognition techniques (such as pattern recognition techniques) to recognize and detect the characteristics of image information (characters, patterns, symbols, etc.) from the images in the application image data, and determine the attribute of the application document indicated by the application image data.

[0045] Although it will be described in detail later, the form of the application image data differs depending on the attribute of the application document. Therefore, when generating the review image data in the image processing unit 30, it is important to perform image processing according to the attribute of the application document. For example, as described above, the application form reading image data 71 (see FIG. 3) is generated by converting the original application form submitted by the applicant into image data. Therefore, the application form reading image data 71 hardly includes image areas other than the application document. On the other hand, the certificate reading image data 72 (see FIG. 4) is generated by converting the copy of the personal identification document submitted by the applicant into image data.

[0046] Therefore, the certificate reading image data 72 may include a margin portion (background image area 721) during printing when creating the copy, or the application documents which are image information may be skewed. In addition, the certificate reading image data may include a plurality of certificate documents as image information in some cases. For example, during examination, when the application form reading image data 71 and the certificate reading image data 72 are displayed on the display device 53 side by side with the examination screen 61, in order for the examiner to refer to the image area corresponding to the examination item on the examination screen 61 (for example, the image area including the image information regarding the "name" of the applicant), it is necessary to adjust the position and inclination of the image by operating an input device 52 such as a mouse. In addition, when the certificate reading image data includes a plurality of certificate documents, there may also be a problem that it is difficult to know which certificate document to refer to. Due to such circumstances, the examination efficiency may be reduced. Therefore, in the information processing apparatus 100, control is required to determine the execution of appropriate image processing according to the attributes of the application documents indicated by the application image data.

[0047] In the present embodiment, the document determination unit 301 determines the attributes of the application documents indicated by the application image data. That is, the image processing unit 30 determines the attributes of the application documents indicated by the application image data in the document determination unit 301, performs image processing on the application image data according to the determined attributes, and generates the examination image data. Thereby, the image processing unit 30 can perform subsequent image processing based on the attributes of the application documents. Therefore, the information processing apparatus 100 which is an image processing apparatus according to the present embodiment can perform control to execute appropriate image processing according to the attributes of the application documents for the application image data indicating various application documents with different formats and printing modes on the paper surface. As a result, it is possible to reduce the labor of displaying the necessary application document data for each confirmation item and improve the examination efficiency. Specifically, by performing image processing according to the attributes determined by the document determination unit 301 on the application image data, it is possible to generate review image data suitable for reviewing the application content (visual review, automatic review) from the application image data, extract information (position information of the review image area described later) for improving the visibility of the review image data on the display device 53 and the reading accuracy of character information on the OCR review server 54. Therefore, the labor of displaying the application document data required for each confirmation item can be reduced, and the review efficiency can be improved.

[0048] For example, in this embodiment, the document determination unit 301 determines whether to divide the application image data based on the determined attributes. For example, the document determination unit 301 determines whether to divide the application image data according to the type of the determined attributes. That is, the document determination unit 301 determines the presence or absence of subsequent image processing (image division processing) based on the attributes of the application documents. Thereby, the information processing apparatus 100 which is an image processing apparatus according to this embodiment performs control to determine appropriate image processing according to the attributes of the application documents, and can surely improve the review efficiency. Specifically, when the attribute of the application document indicated by the application image data is a certificate document (when the application image data is certificate reading image data), the document determination unit 301 determines to divide the application image data. Thereby, the information processing apparatus 100 can perform image division processing (cutting out of the certificate image area 720, inclination correction, etc. described later) on the certificate reading image data, and generate review image data suitable for displaying on the display device 53 side by side with the review screen 61. When the attribute of the application document indicated by the application image data is a certificate document (certificate reading image data), the document determination unit 301 outputs the application image data to the document division unit 302.

[0049] In addition, when the attribute of the application document indicated by the application image data is a certificate document (certificate reading image data), the document determination unit 301 may output the application image data, that is, the certificate reading image data itself, to the examination image storage unit 31 and store it as examination image data. At this time, the document determination unit 301 may store the certificate reading image data and information indicating the attribute of the application document (certificate document) corresponding to the certificate reading image data in the examination image storage unit 31 in association with each other. Thereby, the certificate reading image data with the attribute of the application document can be treated as examination image data that can be displayed by the examination unit 13 on the display device 53. In addition, when the attribute of the application document indicated by the application image data is not a certificate document but an application form (application form reading image data), the document determination unit 301 may output the application image data, that is, the application form reading image data itself, to the examination image storage unit 31 and store it as examination image data. At this time, the document determination unit 301 may store the application form reading image data and information indicating the attribute of the application document (application form) corresponding to the application form reading image data in the examination image storage unit 31 in association with each other. Thereby, the application image data that does not require image segmentation processing, that is, the application form reading image data with the attribute of the application document, can be treated as examination image data that can be displayed by the examination unit 13 on the display device 53.

[0050] That is, the document determination unit 301 may add information indicating the attribute of the application document (application form or certificate document) included in the application image data (application form reading image data, certificate reading image data) to the application image data and store it in the examination image storage unit 31. For example, the information indicating the attribute of the application document may be an identification code (number or character string) indicating the attribute of the application document (application form, certificate document) indicated by the application image data, or may be a file name indicating the attribute.

[0051] In this way, when the document determination unit 301 determines the attributes of the application document indicated by the application image data, it may attach (associate) information (identification code, file name, etc.) indicating the above attributes to the application image data and store it in the examination image storage unit 31. Thereby, the examination unit 13 can acquire the application image data as examination image data from the examination image storage unit 31 based on the information indicating the above attributes attached to the application image data, and the examination efficiency can be improved. Also, in the image processing unit 30, the document division unit 302 and the region extraction unit 303 that perform subsequent image processing after the above attribute determination by the document determination unit 301 can acquire the application image data (examination image data) to which the information indicating the above attributes is attached from the examination image storage unit 31, and perform appropriate image processing according to the above attributes indicated by the application image data.

[0052] Also, the document determination unit 301 may output information (identification code, file name, etc.) representing the above attributes attached to the application image data to the storage unit 12, and store the information indicating the attributes in the storage unit 12 in association with the application image data acquired by the application information acquisition unit 11 and the applicant information indicating the applicant. That is, in the information processing apparatus 100, the application image data stored in the storage area (storage unit 12, examination image storage unit 31) to which the information indicating the above attributes is attached may be treated as examination image data.

[0053] (Document Division Unit) Returning to FIG. 2, the document division unit 302 divides the application image data input from the document determination unit 301 to generate examination image data. Specifically, the document division unit 302 executes image division processing on the certificate reading image data to divide the image area in the certificate reading image data 72 into a certificate image area 720 indicating the certificate document and other image areas. Thereby, the document division unit 302 generates, as examination image data, the certificate image data (certificate image data 8 described later) formed by the certificate image area 720 as examination image data.

[0054] Here, an example of the image segmentation process, which is the image processing executed by the document segmentation unit 302, will be described with reference to FIG. 5. FIG. 5 is a schematic diagram showing an example of the generation of certificate image data in the information processing apparatus 100, which is an image processing apparatus according to the present embodiment. In this example, a method of generating certificate image data by performing an image segmentation process on the certificate reading image data 72 will be described.

[0055] First, the cutting process in the image segmentation process will be described. The document segmentation unit 302 cuts out a certificate image area 720 (certificate image areas 720a and 720b) indicating a certificate from the image area of the application image data (certificate reading image data 72) whose attribute is "certificate document", and generates certificate image data 8 including the image information in the certificate image area 720 as the image data for review.

[0056] That is, the document segmentation unit 302 can generate image data excluding the image area not related to the review (for example, the background image area 721) as the image data for review. For this reason, the load of the work of image adjustment (such as a mouse operation for adjusting the position of the image) by the review person can be reduced. Therefore, the information processing apparatus 100, which is an image processing apparatus according to the present embodiment, can improve the review efficiency.

[0057] The document segmentation unit 302 discriminates between the certificate image area 720 and other areas (for example, the background image area 721) by performing image recognition on the certificate reading image data 72, and specifies the position of the certificate image area 720 in the area within the certificate reading image data 72.

[0058] The document segmentation unit 302 may extract the outer periphery (outline) of the certificate image area 720 by performing edge detection as image recognition, for example, and extract the certificate image area 720 by dividing the image area in the certificate reading image data 72, which is the application image data, into the certificate image area 720 and the background image area 721. Here, the "division" does not mean physical separation (cutting out) of the image data, but means "recognizing the boundary".

[0059] Note that the image recognition process for discriminating (locating) the certificate image area 720 is not limited to edge detection. For example, the document division unit 302 may determine the foreground / background for the image areas in the certificate reading image data 72, and discriminate the foreground image area as the certificate image area 720 and the background image area as the blank background image area 721. In addition to this, the document division unit 302 can also discriminate the certificate image area 720 by various known image recognition techniques.

[0060] When the document division unit 302 generates the certificate image data 8, it outputs and stores the generated certificate image data 8 in the examination image storage unit 31. Thereby, the examination unit 13 can display the examination screen 61 and the certificate image data 8 side by side on the display device 53.

[0061] In addition, the document division unit 302 separately cuts out a plurality of certificate image areas 720 included in the image area of the certificate reading image data to generate a plurality of certificate image data 8. Thereby, the images of a plurality of certificates included in one certificate reading image data can be displayed on the display device 53 as separate examination image data. Therefore, compared with the case where the image information of a plurality of certificates is displayed on the display device 53 as one certificate reading image data, the workload of image adjustment by the examiner can be reduced. Therefore, the information processing apparatus 100, which is an image processing apparatus according to the present embodiment, can improve the examination efficiency.

[0062] In this example, two certificate image regions 720 (certificate image regions 720a and 720b) are included in the image region within the certificate reading image data 72. The document splitting unit 302 separately cuts out the certificate image regions 720a and 720b from the image region within the certificate reading image data 72 in the cutting process. Thereby, separate certificate image data 8a and 8b can be generated from a plurality of certificates included in one certificate reading image data. The document splitting unit 302 repeatedly executes, for example, the discrimination of the certificate image region 720 by the above-described image recognition, extracts (identifies) the positions of all of the plurality of certificate image regions 720 included in the image region within the certificate reading image data 72, and can cut them out from the certificate reading image data 72.

[0063] Thus, when a plurality of certificate image regions 720 are included in the image region of the certificate reading image data, the document splitting unit 302 can separately cut out the plurality of certificate image regions 720 included in the image region of the certificate reading image data to generate a plurality of certificate image data. Also, not limited to this, the document splitting unit 302 may separately cut out the plurality of certificate image regions 720 included in the image region of the certificate reading image data and generate certificate image data for one (specific) certificate image region 720 among the cut-out plurality of certificate image regions 720. That is, the document splitting unit 302 can separately cut out the plurality of the certificate image regions included in the image region of the application image data to generate single or plural certificate image data.

[0064] Also, when a plurality of certificate image regions 720 are included in the image region of the certificate reading image data, the document splitting unit 302 may cut out a specific (for example, one) certificate image region 720 (for example, the certificate image region 720a) among the plurality of certificate image regions 720 to generate the certificate image data 8a. That is, it is not necessary to cut out all of the plurality of certificate image regions 720 included in the image region of the certificate reading image data. In this example, the case where a plurality of certificate image areas 720 are included in the image area of the certificate reading image data has been described as an example. However, when a single certificate image area 720 is included in the image area of the certificate reading image data (for example, when only the certificate image area 720a is included), the document splitting unit 302 may cut out the single certificate image area 720 from the image area of the certificate reading image data to generate the certificate image data 8a.

[0065] Next, the inclination correction process in the image splitting process will be described. For example, when the certificate documents are in an inclined state during printing (for example, the original of the certificate documents is rotated horizontally with respect to the outer periphery of the document placement surface (document set unit) of the printing machine), they may be supplied (placed) to the printing machine. At this time, as shown in FIG. 5, an inclination occurs in the certificate image area 720 indicating the certificate documents (in this example, the certificate image area 720b). Therefore, the document splitting unit 302 performs inclination correction on the generated certificate image data 8. As a result, when the review image data (certificate image data 8) is displayed on the display device 53, the certificate documents indicated by the certificate image data 8 are in a state of facing the review target person directly. For this reason, the visibility of the image information of the certificate documents included in the certificate image data 8 is improved. Therefore, the information processing device 100, which is an image processing device according to the present embodiment, can improve the review efficiency.

[0066] For example, when the document splitting unit 302 reads the image information in the certificate image data 8 by image recognition and detects an inclination in the image information, the document splitting unit 302 may perform rotation correction on the certificate image data so as to make the image information face directly. Note that the document splitting unit 302 may execute the inclination correction process before cutting out the certificate image area 720 from the certificate reading image data 72, and the inclination may be corrected, or the inclination correction may be performed on the certificate image data 8 generated from the certificate reading image data 72. When performing inclination correction on the certificate image area 720, the document splitting unit 302 detects the inclination of the certificate image area 720 in the certificate reading image data 72. If an inclination occurs, the inclination is corrected, and extraction is performed from the image area in the certificate reading image data 72. For example, the document splitting unit 302 may derive the inclination angle of the outer edge 720c of the certificate image area 720 with respect to the outer edge of the image area in the certificate reading image data 72 (for example, the outer edge of the original document placement surface of the printing machine), and rotate the certificate image area 720 based on the inclination angle. Thereby, the inclination of the certificate image area 720 can be corrected.

[0067] Further, the document splitting unit 302 may detect a character string from the image area within the certificate image area 720, and calculate the inclination angle of the certificate image area 720 from the inclination of the character string. The character string to be detected may be a plurality of characters or one character. Note that the document splitting unit 302 may calculate the inclination angle based on the inclination of the character string for the certificate image data 8 generated by cutting out from the certificate reading image data 72, and perform inclination correction on the certificate image data 8 based on the inclination angle.

[0068] Also, the document splitting unit 302 classifies the type of the certificate document (for example, insurance certificate, driver's license, etc.) indicated by the generated certificate image data 8. Thereby, image processing according to the type of the certificate document can be performed in the subsequent processing. For example, the document splitting unit 302 classifies the type of the certificate document based on the type area 722 in the certificate image data 8. The document splitting unit 302 may specify the position of the type area 722 in the certificate image data 8 by, for example, image recognition, and extract image information (for example, character information) indicating the type of the certificate document. For specifying the position of the type area 722, the determination image data (template image of the application document) used for determining the attributes of the application document by the document determination unit 301 may be used.

[0069] Based on the image information ("insurance certificate", "driver's license") included in the type area 722, the document division unit 302 may classify the type of the certificate document indicated by the certificate image data 8a as "insurance certificate" and classify the type of the certificate document indicated by the certificate image data 8b as "driver's license".

[0070] When classifying the type of the certificate document indicated by the certificate image data 8, the document division unit 302 outputs the type of the certificate document to the storage unit 12. As a result, in the storage unit 12, the type of the certificate document indicated by the certificate image data 8 is stored in association with at least one of the application image data acquired by the application information acquisition unit 11 and the applicant information indicating the applicant. Further, the document division unit 302 may output the type of the certificate document indicated by the classified certificate image data 8 to the area extraction unit 303 that performs the subsequent image processing (area extraction processing).

[0071] (Area Extraction Unit) Returning to FIG. 2, the area extraction unit 303 extracts the position information of the image area (review image area) including the image information that is the subject of the review regarding the content of the application document within the image area of the review image data. In the present embodiment, the image information that is the subject of the review (review image information) among the image information in the review image data differs according to the review content (the application content of the applicant). The information processing apparatus 100, which is an image processing apparatus according to the present embodiment, performs an area extraction process in the image processing unit 30 to extract in advance the positions of the item areas and the content areas corresponding to the item areas in the review image data. As a result, it is possible to easily identify the image areas to be reviewed during visual review or automatic review, and the review efficiency can be improved.

[0072] In this embodiment, the area extraction unit 303 can execute a primary extraction process and a secondary extraction process as area extraction processes for extracting the position of the review image area. Here, with reference to FIGS. 3 and 5, an example of the area extraction process (primary extraction process, secondary extraction process) will be described using FIGS. 6 to 8. FIG. 6 is a diagram showing an example of the review image area extracted in the primary extraction process. FIG. 7 is a diagram showing an example of the review image area extracted in the secondary extraction process. FIG. 8 is a diagram showing in more detail the image information included in the certificate image data 8b.

[0073] (Primary extraction process) In the visual review by displaying the review screen 61, the review staff visually checks the image information (visual review image information) of the review image data corresponding to the review items (questions for reviewing the contents of the application documents) displayed on the review screen. Therefore, the area extraction unit 303 extracts the position information of the item area indicating the items of the visual review image information and the position information of the visual content area indicating the content of the visual review image information as the position information of the visual review image area including the visual review image information. As a result, during the visual review, the review unit 13 can display the review image data on the display device 53 in a manner that makes it easy to visually recognize the review image area corresponding to the review items. In addition, since the item area is included in the review image area, the review staff can easily associate the review items with the visual review image information. For this reason, the review efficiency can be improved.

[0074] In addition, the area extraction unit 303 extracts information including the coordinates of the visual review image area within the image area of the review image data as the position information of the visual review image area. As a result, the position of the visual review image area can be specified with higher accuracy within the review image data, and the visibility of the visual review image area can be further improved on the display device 53.

[0075] Here, the extraction of the position information of the visual review image area will be specifically described. First, an example of extracting the position information of the visual review image area in the certificate image data 8 will be described. FIG. 6(a) is a diagram showing an excerpt of a part of the image area of the certificate image data 8a shown in FIG. 5. For example, the area extraction unit 303 extracts the position information of the visual inspection image area 80a including visual inspection image information (here, the name indicated by the insurance certificate, which is the applicant's identity document) within the image area of the certificate image data 8a, which is the image data for examination. Specifically, in the primary extraction process, the area extraction unit 303 extracts the position information of the certificate item area 723 and the position information of the certificate content area 724 within the image area of the certificate image data 8a as the position information of the visual inspection image area 80a.

[0076] FIG. 6(b) is a diagram showing an excerpt of a part of the image area of the certificate image data 8b shown in FIG. 5. For example, the area extraction unit 303 extracts the position information of the visual inspection image area 80b including visual inspection image information (here, the name indicated by the driver's license, which is the applicant's identity document) within the image area of the certificate image data 8b, which is the image data for examination. Specifically, in the primary extraction process, the area extraction unit 303 extracts the position information of the certificate item area 723 and the position information of the certificate content area 724 within the image area of the certificate image data 8b as the position information of the visual inspection image area 80b.

[0077] The application documents shown by the image data for examination are classified into standard documents, semi-standard documents, and non-standard documents according to the regularity of the arrangement (format) of the described items. Standard documents are documents with a fixed format, semi-standard documents are documents with a high degree of regularity although the format is not fixed, and non-standard documents are documents with a low degree of format regularity.

[0078] The certificate image data 8a is classified by the document splitting unit 302 as having the document type of "insurance certificate" and corresponds to semi-standard documents. In the primary extraction process, the area extraction unit 303 extracts the coordinates of the visual inspection image area 80a within the image area of the certificate image data 8a by a process corresponding to semi-standard documents.

[0079] Specifically, the area extraction unit 303 extracts the feature amounts of the certificate image data 8a by a known image recognition technique, and extracts the coordinate information of the certification item area 723 in the certificate image data 8a based on the feature amounts. Further, the area extraction unit 303 acquires the association information between the certification item area 723 and the certification content area 724 from the setting file. The setting file is stored, for example, in the examination image storage unit 31, and the association information includes the position (up and down, left and right) of the content area based on the certification item area 723 and the size of the content area (the number of pixels in the width direction and the height direction). The area extraction unit 303 extracts the coordinates of the certification content area 724 based on the association information. Thereby, the area extraction unit 303 can extract the coordinates of the visual examination image area 80a in the certificate image data 8a. The area extraction unit 303 stores the extracted coordinates of the visual examination image area 80a in the storage unit 12.

[0080] Also, the certificate image data 8b is classified by the document splitting unit 302 as a document of the type "driver's license" and corresponds to a standard document. The area extraction unit 303 extracts the coordinates of the visual examination image area 80b in the image area of the certificate image data 8b by a process according to the standard document in the primary extraction process.

[0081] Since the standard document has a fixed format as described above, the coordinate information of the certification item area and the certification content area of the certificate image data 8b is stored in the storage unit 12 in advance. When the type of the document input from the document splitting unit 302 is "driver's license", the area extraction unit 303 extracts the coordinate information of the certification item area 723 and the certification content area 724 from the storage unit 12. Thereby, the area extraction unit 303 can extract the coordinate information of the visual examination image area 80b. The area extraction unit 303 stores the extracted coordinates of the visual examination image area 80b in the storage unit 12. Note that the area extraction unit 303 may extract the coordinates of the visual examination image area 80b by performing a known image recognition process on the standard document.

[0082] Next, an example of extracting the position information of the visual examination image area in the application form reading image data 71 among the examination image data will be described. FIG. 6(c) is a diagram showing an excerpt of a part of the image area of the application form reading image data 71 shown in FIG. 3. For example, the area extraction unit 303 extracts the position information of the visual inspection image area 80c including visual inspection image information (here, the name described by the applicant) within the image area of the application form reading image data 71 which is the image data for examination. Specifically, in the primary extraction process, the area extraction unit 303 extracts the position information of the application item area 711 and the position information of the application content area 712 within the image area of the application form reading image data 71 as the position information of the visual inspection image area 80c.

[0083] The application form reading image data 71 has been determined by the document determination unit 301 that the attribute of the document is an "application form", which corresponds to an irregular document. In the primary extraction process, the area extraction unit 303 extracts the coordinates of the visual inspection image area 80c within the image area of the application form reading image data 71 by a process corresponding to the irregular document. Specifically, the area extraction unit 303 reads the image information of the application form reading image data 71 as character information by a character recognition technology (for example, OCR) for the image data, and based on the character information and predetermined conditions (such as the number of repetitions of the content area for the item area, and the pattern of the position information where the item area may exist within the application form reading image data 71, etc.), extracts the coordinates of the application item area 711 and the application content area 712. Thereby, the area extraction unit 303 can extract the coordinate information of the visual inspection image area 80c. The area extraction unit 303 stores the extracted coordinates of the visual inspection image area 80c in the storage unit 12. Note that the application form shown by the application form reading image data 71 may be a regular document with a fixed format. In this case, the coordinates of the visual inspection image area 80c within the image area of the application form reading image data 71 can be extracted by the process corresponding to the above-mentioned regular document.

[0084] In this way, in the primary extraction process, the area extraction unit 303 extracts the position information of the visual inspection image area 80 including the visual image information to be examined within the image area of the inspection image data (certificate image data 8, application form read image data 71). Further, as the position information of the visual inspection image area 80, the area extraction unit 303 extracts the position information of the item area (certificate item area 723, application item area 711) indicating the items of the visual image information and the position information of the content area (certificate content area 724, application content area 712) indicating the content of the image information. Also, the area extraction unit 303 extracts information including the coordinates of the visual inspection image area 80 within the image area of the inspection image data as the position information of the visual inspection image area 80. Thereby, the information processing apparatus 100, which is an image processing apparatus according to the present embodiment, can improve the visibility of the visual inspection image area 80 corresponding to the inspection items on the inspection screen 61 in the display device 53 and improve the inspection efficiency.

[0085] (Secondary extraction process) In the secondary extraction process, the area extraction unit 303 extracts the position information of the automatic inspection image area including the image information to be automatically inspected within the image area of the inspection image data. In the automatic inspection in the OCR inspection server 54, OCR processing is executed on the image information of the content area in the inspection image data, and the processing result is output as digital data (for example, text data), and the inspection by the automatic inspection program is performed using the digital data. For example, in the OCR inspection server 54, the image information "Japan Taro" in the certificate content area 724 of the certificate image data 8b and the image information "Japan Taro" in the application content area 712 of the application form read image data 71 are converted into text data by OCR processing. Also, for example, in the automatic inspection program, the text data of each of the certificate content area 724 and the application content area 712 is compared to determine whether they match, and it is determined whether the name described in the application form matches the name indicated by the driver's license.

[0086] In the secondary extraction process, the region extraction unit 303 extracts, as the position information of the automatic review image region in the review image data, the position information of the content region indicating the content of the automatic review image information to be the object of automatic review. Thereby, the image regions (item regions) that are not the objects of automatic review are removed, and only the image regions (content regions) that are the objects of automatic review can be output to the OCR review server 54 in the OCR review server 54. For this reason, compared with the case of performing OCR processing on the entire review image data such as the application form reading image data 71 and the certificate image data 8, the reading accuracy of character information in the OCR processing is improved, and the efficiency and accuracy of automatic review can be improved.

[0087] Also, the region extraction unit 303 extracts, as the position information of the automatic review image region, information including the coordinates of the automatic review image region in the image region of the review image data. Thereby, the position of the automatic review image region is specified with higher accuracy in the review image data, and in the OCR review server 54, the reading accuracy of character information in the OCR processing is further improved, and the efficiency and accuracy of automatic review can be further improved.

[0088] In the secondary extraction process, the region extraction unit 303 extracts, as the position information of the automatic review image region, the position information of the content region excluding the position information of the item region from the visual review image region 80 extracted in the primary extraction process. Here, although the secondary extraction process is described as an example of the process of extracting the position information of the content region, when the application documents indicated by the review image data do not include item regions, the process of directly extracting the content region may be performed as described later. Also, for example, when extracting the content region from the review image data by image recognition processing (AI image processing) using AI technology as described later, the process of directly extracting the content region can also be performed. Depending on the characteristics of the AI image processing, etc., the extraction accuracy may be improved by directly extracting the content region from the review image data. Thus, in the present disclosure, the region extraction process in which the image processing unit 30 (region extraction unit 303) extracts the position of the review image region may be a process composed of the primary extraction process and the secondary extraction process as described above, or may be a process of directly extracting the content region from the review image data. Here, the secondary extraction process will be specifically described below.

[0089] For example, the region extraction unit 303 extracts the position information excluding the position information of the certification item region 723 from the visual inspection image region 80a shown in FIG. 6(a). As a result, as shown in FIG. 7(a), the position information including the coordinates of the automatic inspection image region 81a formed by the certification content region 724 of the certificate image data 8a can be extracted. Also, the region extraction unit 303 extracts the position information excluding the position information of the certification item region 723 from the visual inspection image region 80b shown in FIG. 6(b). As a result, as shown in FIG. 7(b), the position information including the coordinates of the automatic inspection image region 81b formed by the certification content region 724 of the certificate image data 8b can be extracted.

[0090] In addition, the region extraction unit 303 extracts the position information excluding the position information of the application item region 711 from the visual inspection image region 80c shown in FIG. 6(c). As a result, as shown in FIG. 7(c), the position information including the coordinates of the automatic inspection image region 81c formed by the application content region 712 of the application form reading image data 71 can be extracted.

[0091] In this way, in the secondary extraction process, the region extraction unit 303 extracts the position information of the content region (certification content region 724, application content region 712) indicating the content of the automatic inspection image information that is the object of automatic inspection as the position information of the automatic inspection image region 81 in the inspection image data. Thereby, the reading accuracy of the character information in the OCR process is improved, and the efficiency and accuracy of the automatic inspection can be improved. Note that when the information processing apparatus 100 is not connected to the OCR inspection server 54, the region extraction unit 303 does not need to execute the secondary extraction process. That is, in the information processing apparatus 100, the region extraction unit 303 may have a configuration capable of executing at least the primary extraction process.

[0092] Note that the coordinate information of the review image areas (visual review image area 80, automatic review image area 81) extracted by the area extraction unit 303 may be the coordinate information (X, Y) of each corner (in this example, a rectangle) of the review image area. Further, it may be composed of the coordinate information (X, Y) of one corner (for example, the upper left corner) of the review image area and the size information of the review image area based on the corner. The size information only needs to include the width W (number of pixels in the width direction) and the height H (number of pixels in the height direction).

[0093] FIG. 8 is a schematic diagram showing the image area of the certificate image data 8b in more detail. Inside the image area of the certificate image data 8b showing the driver's license, a plurality of certificate item areas 723 (certificate item areas 723a to 723f) are included as certificate item areas including image information indicating the types of various information regarding the applicant and the certificate content. Further, inside the image area of the certificate image data 8b, a plurality of certificate content areas 724 (certificate content areas 724a to 724f) are included as certificate content areas including image information indicating the content of the information of the type indicated by the certificate item area. Each of the certificate item areas 723a, 723b, 723c, 723d, 723e, 723f corresponds to each of the certificate content areas 724a, 724b, 724c, 724d, 724e, 724f, respectively.

[0094] In the area extraction process, the area extraction unit 303 can extract, as a visual review image area, an area including the certificate item areas 723 (certificate item areas 723a to 723f) and the certificate content areas 724 (certificate content areas 724a to 724f) within the image area of the certificate image data 8b. That is, the area extraction unit 303 can extract the position information of a plurality of visual review image areas from the review image data by repeatedly executing the above-described coordinate information extraction process. Thereby, for each of the plurality of review items displayed on the review screen 61 as the review progresses during display on the display device 53 in the visual review, the visibility of the corresponding visual review image area can be improved.

[0095] (Area division unit) Based on the position information (coordinate information) of the review image area extracted by the area extraction unit 303, the area division unit 304 cuts out the review image area from within the image area of the review image data, and generates area division image data formed by the review image area as new review image data. For example, based on the position information (coordinate information) of the automatic review image area 81 extracted by the area extraction unit 303, the area division unit 304 cuts out the automatic review image area 81 from within the image area of the review image data, and generates area division image data formed by the automatic review image area 81 as new review image data. The area division unit 304 stores the generated area division image in the review image storage unit 31. Thereby, the data output unit 15 outputs the area division image to the OCR review server 54. In this case, the OCR review server 54 may target the area division image including the automatic review image area 81 for OCR processing, and there is no need to extract the automatic review image area 81 from the review image data. Thereby, the reading accuracy of character information in the OCR processing is improved, and the efficiency and accuracy of the automatic review are further improved.

[0096] Also, based on the position information (coordinate information) of the visual review image area 80 extracted by the area extraction unit 303, the area division unit 304 may cut out the visual review image area 80 from within the image area of the review image data, and generate area division image data formed by the visual review image area 80 as new review image data. In this case, the review unit 13 can display only the visual review image area 80 of the review image data on the display device 53 together with the review screen. Thereby, the visibility of the image information to be reviewed is further improved, and the review efficiency can be further improved.

[0097] Note that regardless of whether the information processing apparatus 100 is connected to the OCR review server, the region division unit 304 is not an essential component in the image processing unit 30. That is, the generation of region division image data is not essential. For example, the information processing apparatus 100 can perform highly efficient and highly accurate automatic review by outputting a combination of review image data (e.g., application form reading image data 71, certificate image data 8) and the position information (coordinates) of the automatic review image region 81 extracted by the region extraction unit 303 to the OCR review server 54. Note that when the image processing unit 30 has the region division unit 304, it may only execute the extraction of the visual review image region 80 and store the extracted visual review image region 80 in the review image storage unit 31. In this case, the review unit 13 may be configured to display the extracted visual review image region 80 on the display device 53.

[0098] As described above, the image processing unit 30 performs image processing to generate at least review image data (certificate image data 8, region division image by the automatic review image region 81) for visual review based on the application image data and stores it in the review image storage unit 31. Thereby, the review unit 13 can display the review image data on the display device 53 together with the review screen. Note that the process of the document determination unit 301 storing (memorizing) the application image data indicating the application form in the review image storage unit 31 also corresponds to the generation of the review image data.

[0099] FIG. 9 shows an example of the review screen 61 displayed on the display screen of the display device 53. On the display screen of the display device 53, the review screen 61 and the review image data are displayed during visual inspection. On the review screen 61, review items (questions) for the review image data are displayed. In this example, as the review image data corresponding to the review item (name in the personal identification document), the certificate image data 8b is displayed. As described above, the image processing unit 30 (particularly the region extraction unit 303) extracts the position information of the visual inspection image region 80 (in this example, the visual inspection image region 80b) by the primary extraction process and stores it in the storage unit 12. Specifically, in the storage unit 12, the position information of the visual inspection image region 80 is stored in association with the data indicating the storage destination of the review image data and the review item. Therefore, the review unit 13 can specify the position of the visual inspection image region 80 in the review image data when the review screen 61 is displayed, and display the visual inspection image region 80 in a highly visible manner. Thereby, the information processing device 100 which is an image processing device according to the present embodiment can improve the review efficiency.

[0100] In this example, the review unit 13 enlarges and displays on the display device 53 the visual inspection image region 80 including the image information corresponding to the review item in the review screen 61. Note that the display mode of the visual inspection image region 80 is not limited to enlargement display, and may be highlighting display (display in a mode different from other image regions in terms of display mode (color, font, outer frame, etc.)).

[0101] Also, as shown in FIG. 10, the review unit 13 can display a plurality of types of review image data as the review image data corresponding to the review items (name in the personal identification document and name in the application form) related to a plurality of application documents on the display device 53. In this example, as the review image data corresponding to the review item, the certificate image data 8b and the application form read image data 71 are displayed. Also in this case, the visual inspection image regions 80 (in this example, the visual inspection image regions 80b, 80c) can be displayed in a highly visible manner (for example, enlargement display). Thereby, the information processing device 100 which is an image processing device according to the present embodiment can further improve the review efficiency.

[0102] (1.2) Hardware Configuration of the Information Processing Apparatus FIG. 11 is a schematic diagram showing an example of the hardware configuration in the functional units of the information processing apparatus 100. As shown in FIG. 11, the information processing system 1 is realized by the arithmetic unit 10. The arithmetic unit 10 is an information processing apparatus (computer) including a processor 101, a ROM (Read Only Memory) 102, a RAM (Random access memory) 103, and an auxiliary storage device (memory) 104. Connected to the arithmetic unit 10 are, for example, a reading device 51, an input device 52, a display device 53, and an external storage medium 57 via an external interface (I / F) 105. The external interface (I / F) 105 is, for example, a USB (Universal Serial Bus) port or the like.

[0103] Another computer 55 may be connected to the arithmetic unit 10 via, for example, a communication interface (I / F) 106 and a network 108. The communication interface (I / F) 106 is, for example, a LAN (Local Area Network) port, a wireless LAN interface, or the like. Here, the reading device 51, the input device 52, and the display device 53 are the reading device 51, the input device 52, and the display device 53 connected to the information processing apparatus 100 shown in FIG. 2. Also, the reading device 51, the input device 52, the display device 53, and the computer 55 are provided with a user interface (not shown) for connecting to the arithmetic unit 10. Here, when the hardware resources of each part of the arithmetic unit 10 cooperate with a predetermined program, the functions of the application information acquisition unit 11, the storage unit 12, the examination unit 13, the image processing unit 30, the examination image storage unit 31, the display data output unit 14, and the data output unit 15 of the information processing apparatus 100 are constructed.

[0104] Processor 101 is a processor that controls the entire information processing apparatus 100. Processor 101 reads out the program stored in ROM 102 via bus 107 and controls information processing apparatus 100 according to the program. Processor 101 controls information processing apparatus 100 so as to perform review processing such as outputting a predetermined review screen of information processing apparatus 100. ROM 102 stores an image processing program that executes image processing for generating review image data, and a program that executes an application information processing method for receiving an application procedure and obtaining the result of review by a reviewer. RAM 103 temporarily stores calculation data, various data input by a reviewer via input device 52, and the like.

[0105] Auxiliary storage device 104 is backed up by, for example, a battery (not shown), and the storage state is maintained even when the power of information processing system 1 is turned off. The auxiliary storage device 104 stores the image data of the application documents input from the reading device 51 via the external interface 105. In addition to the input data input by the reviewer from the input device 52, the auxiliary storage device 104 stores various data input via the external interface 105 and the communication interface 106.

[0106] Reading device 51 converts application documents into electronic data (image data) and inputs the image data to arithmetic unit 10 via external interface 105. The control of reading by reading device 51 may be performed by arithmetic unit 10 of information processing system 1 executing a program, or may be performed by control from another device.

[0107] Input device 52 is a keyboard, card reader, pointing device, or voice input device such as a microphone, etc., and inputs commands and input data based on the operations of the reviewer to arithmetic unit 10 via external interface 105. The display device 53 is, for example, a liquid crystal display (LCD) or an organic EL display (OELD), and receives and displays various data from the arithmetic unit 10 via the external interface 105. The display device 53 displays, for example, each data read onto the memory of the arithmetic unit 10, data obtained as a result of execution of a program or the like, review image data based on image data of application documents scanned by the reading device 51, data output during the review process in the review unit 13, and the like.

[0108] The external storage medium 57 is, for example, an optical disk such as a CD (Compact Disk), a DVD (Digital Versatile Disk), or a Blu-ray Disc (registered trademark), a semiconductor memory such as a flash memory, or a portable hard disk drive. The external storage medium 57 may store the review data output from the information processing apparatus 100.

[0109] The computer 55 is connected to the arithmetic unit 10 via the network 108 through the communication interface 106. The computer 55 includes a display device and an input device (not shown), and can input commands and input data based on the operations of the operator to the arithmetic unit 10 via the input device through the network 108, or receive various data from the arithmetic unit 10 and display them on the display device. The network 108 may be either a wired network or a wireless network. When the network 108 is a wireless network such as a wireless LAN, Bluetooth (registered trademark), or a mobile communication network, the information processing apparatus 100 can receive commands in real time from a review person located at a remote location, or display the execution status of a program or the like and the acquired data to the review person located at a remote location. For example, when the computer 55 is a local government computer and the information processing apparatus 100 is executed on a computer of a company commissioned by the local government for review, the review data is output from the information processing system 1 and transmitted to the local government computer 55 via the network 108.

[0110] (1.3) Image processing method Hereinafter, an example of the information processing method according to the present embodiment will be described with reference to FIG. 12. FIG. 12 is a flowchart for explaining an example of the image processing method according to the present embodiment. When the application information acquisition unit 11 acquires application image data from the reading device 51 and stores it in the storage unit 12, the document determination unit 301 of the image processing unit 30 reads the application image data from the storage unit 12 (step S101). For example, the document determination unit 301 reads application image data that is not associated with storage destination data indicating the storage destination of the review image data (application image data for which the corresponding review image data has not been generated). Next, the document determination unit 301 determines the attributes of the application document indicated by the read application image data (step S102). Subsequently, the document determination unit 301 determines whether to divide the application image data based on the attributes determined in step S102 (step S103). When the determined attribute is "certificate document", the document determination unit 301 determines to divide the application image data and outputs the application image data to the document division unit 302. Also, when the determined attribute is not "certificate document", that is, when it is an "application form", the document determination unit 301 stores the application image data in the review image storage unit 31 and further outputs it to the area extraction unit 303.

[0111] Based on the input of the application image data, the document division unit 302 of the image processing unit 30 extracts the certificate image data 8 from the application image data (step S104). Subsequently, the document division unit 302 corrects the inclination of the certificate image data 8 (step S105), then classifies the type of the certificate document indicated by the certificate image data 8 (step S106), and outputs the attributes of the certificate image data to the area extraction unit 303 of the image processing unit 30.

[0112] Based on the input of the type of the certificate document from the document splitting unit 302 or the input of the application image data from the document determination unit 301 (NO in step S103), the area extraction unit 303 extracts the position information of the review image areas (visual review image area 80, automatic review image area 81) within the image area of the review image data (step S107). When storing the position information of the review image area in the storage unit 12, the area extraction unit 303 outputs the position information of the review image area to the area splitting unit 304. Based on this, the area splitting unit 304 cuts out the review image area from the review image data and stores it in the review image storage unit 31.

[0113] As described above, in the image processing method according to the present embodiment, application image data, which is image data of an application document including at least one attribute of an application form containing applicant's application information and the applicant's certificate document among the application documents submitted by the applicant, is acquired (step S101), the attribute of the application document indicated by the application image data is determined (step S102), and image processing is performed on the application image data according to the attribute to generate review image data used for reviewing the content of the application information (steps S103, S104, S108). Thereby, for example, control of image processing such as determining whether to split the application image data based on the attribute of the application document indicated by the application image data (step S104) becomes possible, review image data can be generated by image processing according to the mode of the application image data, and the review efficiency can be improved.

[0114] (1.4) Image Processing Program An image processing program executed by an information processing apparatus 100, which is an image processing apparatus according to the present embodiment, will be described. The information processing apparatus 100 performs image processing according to a program that causes a computer to execute at least the following operations (a) to (c). The following program is non-temporarily recorded on a recording medium such as a hard disk drive, a memory, or an optical disk such as a DVD disk or a Blu-ray (registered trademark). The following program may be distributed via the Internet. Further, the following program may be recorded in a cloud server and executed via the Internet.

[0115] (a) Obtaining application image data (application form reading image data 71, certificate reading image data 72), which is image data of at least one of an application form containing applicant's application information and the applicant's certificate documents among the application documents submitted by the applicant. (b) Determining whether to divide the application image data based on the attributes of the application documents indicated by the application image data. (c) Performing image processing on the application image data according to the attributes of the application documents to generate review image data used for reviewing the content of the application information (for example, saving the application form reading image data 71 to the review image storage unit 31, cutting out the certificate image area 720, and at least one of cutting out the review image area (visual review image area 80, automatic review image area 81)).

[0116] (1.5) Modification example In the above-described information processing apparatus 100, the image processing in the image processing unit 30 may be executed using AI technology. For example, the image processing unit 30 may have a learned model capable of classifying the attributes of the application documents indicated by the application image data into either an application form or certificate documents. Thereby, the image processing unit 30 can input the application image data into the learning model and determine the attributes of the documents indicated by the application image data based on the classification result. Thereby, the determination accuracy of the attributes can be improved.

[0117] In addition, the image processing unit 30 may have a pre-trained model capable of cropping and correcting the certificate image data 8 based on the application image data indicating the certificate documents and classifying the types of the certificate documents. Thereby, the image processing unit 30 can input the application image data indicating the certificate documents into the learning model and obtain the certificate image data 8 as an output result. Thereby, the accuracy of generating the certificate image data 8 and classifying the types can be improved.

[0118] In addition, the image processing unit 30 may have a pre-trained model capable of extracting the position information (coordinate information) of the review image area from the review image data. Thereby, the image processing unit 30 can input the review image data into the learning model and obtain the position information (coordinate information) of the review image area as an output result. Thereby, the accuracy of extracting the position information can be improved. Further, when extracting the position information by the pre-trained model, the position information of the automatic review image area 81 can be extracted without the procedure of extracting the visual review image area 80 (without executing the primary extraction process). That is, the image processing unit 30 can extract the position information of the content area (application content area 712, certificate content area 724) indicating the content of the image information (automatic review image information) to be reviewed as the position information of the automatic review image area 81. In this way, by using the AI technology, the image processing unit 30 can improve the accuracy of image processing and as a result improve the efficiency of review.

[0119] Note that the image processing unit 30 may directly extract the position information of the content areas (application content area 712, certificate content area 724) from the review image data (e.g., application form reading image data 71, certificate image data 8) without using a learned model (extracting without performing the primary extraction process). For example, when the application documents (certificate documents, application documents) indicated by the application image data are standard documents, the area extraction unit 303 may only extract the coordinate information of the content areas stored in the storage unit 12 in advance. Also, when the application documents indicated by the application image data are semi-standard documents, the coordinate information of the content areas may be set in the setting file, and the area extraction unit 303 may extract the coordinate information of the content areas from the setting file. Further, the area extraction unit 303 may extract the content areas from the review image data by image recognition processing. For example, the area extraction unit 303 may compare the review image data (e.g., application form reading image data 71, certificate image data 8) with the image data of the template (template) of the application documents (judgment image data), and obtain the coordinates of the image area of the review image data corresponding to the content area in the template image, and extract it as the position information of the content area in the review image data.

[0120] Also, in the information processing apparatus 100 described above, the image processing unit 30 (area extraction unit 303) has been extracting the position information (coordinates) of the review image area (at least the visual review image area 80) using the application form reading image data 71 and the certificate image data 8 generated from the certificate reading image data 72 after the attribute determination of the application documents as the review image data. However, the present disclosure is not limited to this. For example, the image processing unit 30 (area extraction unit 303) may extract the position information of the review image area using the application image data (application form reading image data 71, certificate reading image data 72) after the attribute determination of the application documents. That is, in the information processing apparatus 100 of the present disclosure, without generating the review image data (certificate image data 8) as new image data from the application image data, the application image data with the attributes of the application documents is used as the review image data, and the position information of the review image area may be extracted using the review image data. As described above, the information processing apparatus 100 of the present disclosure only needs to be able to determine at least the attributes of the application documents indicated by the application image data. Further, for the application image data to which information indicating the attributes of the application documents is attached, the position information of the image area for examination (at least the visual examination image area 80) may be extracted. That is, the image processing unit 30 in the information processing apparatus 100 only needs to be able to execute at least the reading of the application image data (step S101 above) and the determination of the attributes of the application documents indicated by the application image data (step S102 above), and may be configured to be able to execute the extraction of the image area for examination (step S107 above). Further, a combination of the application image data to which information indicating the attributes of the application documents is attached and the position information of the image area for examination may be treated as the image data for examination.

[0121] In this case, for example, the area extraction unit 303 of the image processing unit 30 may acquire the coordinates of the visual examination image area 80 in the certificate image area 720 in the certificate reading image data 72. The area extraction unit 303 may extract the coordinates of the four corners in the rectangular visual examination area from the image area of the certificate reading image data 72 by, for example, various known image recognition techniques, and store the coordinates in the storage unit 12 in association with the application image data (certificate reading image data 72). Further, for the application form reading image data 71, the coordinates of the visual examination image area 80 may be acquired as described above. Also in this case, the examination unit 13 can specify the position (coordinates) of the visual examination image area 80 in the application image data (application form reading image data 71, certificate reading image data 72) based on the position information of the visual examination image area 80, and display the visual examination image area on the display device 53 in a highly visible manner. Therefore, the examination efficiency can be improved. Similarly, the image processing unit 30 may extract the position information (coordinates) of the automatic examination image area 81 for the application image data to which information indicating the attributes of the application documents is attached, or may further generate area division image data. Also in this case, the OCR examination server 54 can perform highly efficient and highly accurate automatic examination using the combination of the application image data output from the information processing apparatus 100 and the position information of the automatic examination image area 81, or the area division image data.

[0122] The present disclosure has been described above with reference to each embodiment. However, the scope of the present disclosure is not limited to the illustrated and described exemplary embodiments, but also includes all embodiments that bring about equivalent effects to those intended by the present disclosure. Furthermore, the scope of the present disclosure is not limited to the combinations of the features of the invention defined by the claims, but may be defined by any desired combination of the specific features among all the disclosed features respectively.

Explanation of Reference Numerals

[0123] 1 Information processing system 9 Network 11 Application information acquisition unit 12 Storage unit 13 Examination unit 14 Display data output unit 15 Data output unit 30 Image processing unit 31 Examination image storage unit 51 Reading device 52 Input device 53 Display device 54 OCR examination server 100 Information processing device 301 Document determination unit 302 Document splitting unit 303 Region extraction unit 304 Region splitting unit

Claims

1. An image acquisition unit that acquires application image data, which is image data of at least one attribute of an application form and the applicant's certificate documents among the application documents submitted by the applicant and includes the applicant's application information; An image processing unit that determines whether the attribute of the application document indicated by the application image data is the application form or the certificate documents, performs image processing on the application image data according to the attribute, and generates review image data for use in reviewing the content of the application information; The application image data includes certificate copy image data, which is an image data of a paper medium that is a copy of the certificate documents and has a part of the certificate documents printed thereon; The image area of the certificate copy image data includes a certificate image area showing the whole of the certificate documents printed in the paper surface and a background image area showing the margin where the certificate documents are not arranged in the paper surface; The image processing unit has a trained model capable of classifying the attribute of the application document, and determines whether the attribute is the application form or the certificate documents based on the classification result obtained by inputting the application image data into the trained model; Further, the image processing unit As the image processing, it is possible to execute a process of discriminating the certificate image area and the background image area with respect to an image area showing the inside of the paper surface of the certificate copy image data among the application image data, cutting out the certificate image area excluding the background image area from the image area of the certificate copy image data, and generating, as the review image data, certificate image data which is new image data including the certificate image area, and determines whether to execute the image processing based on the determination result of the attribute An image processing apparatus characterized by the above.

2. The image processing unit As the image processing, it is possible to execute a process of correcting the inclination of the certificate image data generated as the review image data The image processing apparatus according to claim 1, characterized by the above.

3. In the image area of the certificate copy image data, a plurality of certificate image areas each showing the whole of the plurality of certificates printed within the paper surface of the one printed matter and the background image area are included, The image processing unit, As the image processing, a plurality of certificate image data are generated as the image data for examination by separately cutting out a plurality of the certificate image areas excluding the background image area from the image area of the certificate copy image data and using the new image data including the certificate image areas. The image processing apparatus according to claim 1 or 2, characterized in that.

4. The image processing includes a boundary recognition process for recognizing a boundary between the certificate image area and the background image area other than the certificate image area in the image area of the certificate copy image data among the application image data, The image processing unit determines whether to execute the boundary recognition process based on the determination result of the attribute of the application document indicated by the application image data. The image processing apparatus according to any one of claims 1 to 3, characterized in that.

5. The image processing unit, When the attribute of the application document indicated by the application image data is the certificate, the boundary recognition process is executed to cut out the certificate image area from the image area of the application image data, and the certificate image data is generated as the new image data that becomes the image data for examination. The image processing apparatus according to claim 4, characterized in that.

6. The image processing unit, Classify the types of the certificates based on the image information included in the certificate image data, and extract the position information of the examination image area including the image information to be examined within the image area of the image data for examination based on the classified types of the certificates or the attribute of the application document. The image processing apparatus according to any one of claims 1 to 5, characterized in that

7. The image processing unit extracts, as the position information of the review image area, information including the coordinates of the review image area within the image area of the review image data The image processing apparatus according to claim 6, characterized in that

8. The image processing unit extracts, as the position information of the review image area, the position information of the item area indicating the item of the image information to be reviewed and the position information of the content area indicating the content of the image information The image processing apparatus according to claim 6 or 7, characterized in that

9. The image processing unit extracts, as the position information of the review image area, the position information of the content area indicating the content of the image information to be reviewed The image processing apparatus according to claim 6 or 7, characterized in that

10. The image processing unit extracts, as the primary extraction information, the position information of the area including the item area indicating the item of the image information to be reviewed and the content area indicating the content of the image information, and extracts, as the position information of the review image area, the position information of the content area obtained by removing the position information of the item area from the primary extraction information The image processing apparatus according to claim 6 or 7, characterized in that

11. The image processing unit cuts out the review image area from the image area of the review image data based on the extracted position information, and generates region-divided image data formed by the review image area as new review image data The image processing apparatus according to any one of claims 6 to 10, characterized in that

12. includes a review image data storage unit for storing the review image data The image processing unit stores the generated examination image data in the examination image data storage unit. The image processing apparatus according to any one of claims 1 to 11, characterized in that.

13. An image processing method executed by a computer, obtaining application image data, which is image data of at least one attribute of an application form and a certificate of the applicant among the application documents submitted by the applicant and includes the applicant's application information, determining whether the attribute of the application document indicated by the application image data is the application form or the certificate, and performing image processing on the application image data according to the attribute to generate examination image data used for examining the content of the application information, The application image data includes certificate copy image data, which is an image data of a paper medium that is a copy of the certificate and has a part of the certificate printed thereon. The image area of the certificate copy image data includes a certificate image area showing the whole of the certificate printed in the paper surface and a background image area showing the margin where the certificate is not arranged in the paper surface. As the image processing, based on the classification result obtained by inputting the application image data into a learned model capable of classifying the attribute of the application document, it is determined whether the attribute is the application form or the certificate. Furthermore, as the image processing, the certificate image area and the background image area are discriminated from the image area showing the paper surface of the certificate copy image data in the application image data, and the certificate image area obtained by removing the background image area from the image area of the certificate copy image data is cut out, and a process of generating certificate image data, which is new image data including the certificate image area, as the examination image data is executable, and it is determined whether to execute the image processing based on the determination result of the attribute. An image processing method characterized by the above.

14. Obtaining application image data, which is image data of at least one attribute of an application form containing applicant's application information and the applicant's certificate documents among the application documents submitted by the applicant; Determining whether the attribute of the application document indicated by the application image data is the application form or the certificate documents; Performing image processing on the application image data according to the attribute to generate review image data for use in reviewing the content of the application information; The application image data includes certificate copy image data, which is an image data of a paper medium that is a copy of the certificate documents and has a part of the certificate documents printed thereon. The image area of the certificate copy image data includes a certificate image area showing the whole of the certificate documents printed in the paper surface and a background image area showing the margin where the certificate documents are not arranged in the paper surface; As the image processing, determining whether the attribute is the application form or the certificate documents based on the classification result obtained by inputting the application image data into a trained model capable of classifying the attribute of the application document; Furthermore, as the image processing, discriminating the certificate image area and the background image area for the image area showing the paper surface of the certificate copy image data among the application image data, and cutting out the certificate image area excluding the background image area from the image area of the certificate copy image data to generate certificate image data, which is new image data including the certificate image area, as the review image data, and determining whether to execute the image processing based on the determination result of the attribute; Causing a computer to execute the above; An image processing program characterized by the above.

Images