Information processing device, information processing method, and program for information processing device

Abstract

The present invention provides an information processing device or the like for improving the accuracy of document sorting. The present invention: acquires a document image of a standard document in which the positions of items are determined with respect to the type of the document; acquires template information about a template image prepared for each type of the document; compares the template information with information about the document image to determine the type of the document (S6, S7); and generates display data for displaying an image including the document image on a terminal device 20 so as to receive information about designating range of masking 50a for a prescribed item (S9).

Description

Information processing device, information processing method, and program for information processing device

[0001] The present invention relates to an information processing device, an information processing method, and a program for the information processing device that processes image data of a document.

[0002] For example, Japanese Patent Laid-Open No. 2007-121299 discloses a program that causes a computer to execute a process of: generating a first component image by extracting horizontally extending line segments from an image of a form and a second component image by extracting vertically extending line segments; dividing the first component image into a plurality of blocks; generating a first feature value indicating a distribution of the line segments in the first component image based on the presence or absence of line segments in each block; dividing the second component image into a plurality of blocks; generating a second feature value indicating a distribution of the line segments in the second component image based on the presence or absence of line segments in each block; and identifying and classifying the type of the form in the image based on a difference between the generated first feature value and a first feature value in a form definition registered in advance in a definition body, and a difference between the generated second feature value and a second feature value in the form definition.

[0003] Japanese Patent Application Laid-Open No. 2022-108130

[0004] However, in operations such as opening a bank account, tasks such as verifying identity documents are required, and the accuracy of the work is of utmost importance, so if there is even the slightest uncertainty in tasks such as sorting documents by type, manual sorting may be necessary. However, as the number of documents that are sent to manual work increases, work efficiency decreases, so there is a need to further improve the accuracy of document sorting, which is one of the work tasks.

[0005] The present invention has been made in view of the above problems, and an object of the present invention is to provide information processing and the like that improves the accuracy of document sorting.

[0006] In order to solve the above problem, the invention described in claim 1 is characterized by comprising: document image acquisition means for acquiring a document image of a standardized document in which the positions of items are determined according to the type of document; template information acquisition means for acquiring template information of a template image prepared for each type of document; standardized document determination means for comparing the template information with information on the document image to determine the type of the document; and display data generation means for receiving information specifying the masking range for a predetermined item and generating display data for displaying an image including the document image on a terminal device.

[0007] Furthermore, the invention described in claim 2 is characterized in that, in the information processing device described in claim 1, when the display data generation means receives information specifying the masking range, it generates the display data in which the area corresponding to the information specifying the range is displayed semi-transparently, and when it receives information confirming the masking, it generates the display data in which the area corresponding to the information specifying the range is filled in.

[0008] According to a third aspect of the present invention, in the information processing device of the first aspect, the template information acquisition means acquires information on feature points of the template image, and the standard document discrimination means extracts feature points from the document image and compares the extracted feature points with feature points of the template image to discriminate the type of the document.

[0009] Furthermore, the invention described in claim 4 is characterized in that, in the information processing device described in claim 3, the feature points of the template image are feature points extracted from an image obtained by removing variable information including character information from the template image.

[0010] Furthermore, the invention described in claim 5 is characterized in that, in the information processing device described in claim 3 or claim 4, the standard document discrimination means extracts feature points corresponding to feature points of the template image from the feature points extracted from the document image, selects feature points from the corresponding feature points according to the Hamming distance between the feature points, and discriminates the type of the document based on the number of selected feature points.

[0011] Furthermore, the invention described in claim 6 is characterized in that, in the information processing device described in claim 5, the standard document discrimination means discriminates the type of the document based on the number of the selected feature points and the number of feature points of the template image.

[0012] Furthermore, the invention described in claim 7 is characterized in that, in the information processing device described in claim 3, the type of the document is determined based on the number of feature points of the document image corresponding to the feature points of the template image.

[0013] Furthermore, the invention described in claim 8 is characterized in that the information processing device described in claim 3 further comprises an image correction means for correcting the document image based on feature points extracted from the document image.

[0014] Furthermore, the invention described in claim 9 is characterized in that, in the information processing device described in claim 1, it further comprises a code reading means for reading a code written on the document from the document image, and the standard document discrimination means compares the read code with a code associated with the template image to determine the type of the document.

[0015] Furthermore, the invention described in claim 10 is characterized in that, in the information processing device described in claim 1 or claim 9, the standard document discrimination means compares the document image with the template image to discriminate the type of the document.

[0016] The invention described in claim 11 is characterized by comprising a document image acquisition means for acquiring a document image of a document including an identification document; a character feature extraction means for extracting character features from text data generated by optically reading the document image; a document discrimination means for discriminating the type of the document based on the extracted character features using a trained machine learning model that outputs a document type when character features are input; and a display data generation means for generating display data for displaying an image including the document image on a terminal device in order to receive information specifying the masking range for a specified item.

[0017] Furthermore, the invention described in claim 12 is characterized in that, in the information processing device described in claim 11, when the display data generation means receives information specifying the masking range, it generates the display data in which the area corresponding to the information specifying the range is displayed semi-transparently, and when it receives information confirming the masking, it generates the display data in which the area corresponding to the information specifying the range is filled in.

[0018] Furthermore, the invention described in claim 13 is characterized in that in the information processing device described in claim 11 or claim 12, the character feature extraction means extracts the character features constituted by character strings obtained by dividing the text data into predetermined character units and the frequency of appearance of each of the divided character strings in the text data, and there are a plurality of the predetermined character units.

[0019] In addition, the invention described in claim 14 is the information processing apparatus described in claim 13, characterized in that at least one of the predetermined character units is a single character unit.

[0020] Furthermore, the invention described in claim 15 is the information processing apparatus described in claim 13, wherein the character feature extraction means extracts the character feature by weighting the appearance frequency of each of the divided character strings.

[0021] The invention described in claim 16 is characterized in that it includes a document image acquiring step in which document image acquiring means acquires a document image of a standard document in which the positions of items are determined for each document type; a template information acquiring step in which template information acquiring means acquires template information of a template image prepared for each document type; a standard document determining step in which standard document determining means compares the template information with information about the document image and then determines the document type; and a display data generating step in which display data generating means generates display data for displaying an image including the document image on a terminal device to receive information specifying a masking range for the predetermined item.

[0022] The invention described in claim 17 is characterized in that it includes a document image acquisition step in which a document image acquisition means acquires a document image of a document including an identification document; a character feature extraction step in which a character feature extraction means extracts character features from text data generated by optically reading the document image; a document discrimination step in which a document discrimination means uses a trained machine learning model that outputs a document type when character features are input to discriminate the type of the document based on the extracted character features; and a display data generation step in which a display data generation means generates display data for displaying an image including the document image on a terminal device in order to accept information specifying the masking range for the specified item.

[0023] The invention described in claim 18 is characterized in that a computer functions as document image acquisition means for acquiring a document image of a standard document in which the positions of items are determined according to the type of document, template information acquisition means for acquiring template information of a template image prepared for each type of document, standard document determination means for comparing the template information with information on the document image to determine the type of document, and display data generation means for receiving information specifying the masking range for a predetermined item and generating display data for displaying an image including the document image on a terminal device.

[0024] The invention described in claim 19 is characterized in that a computer is made to function as a document image acquisition means for acquiring document images of documents including personal identification documents, a character feature extraction means for extracting character features from text data generated by optically reading the document image, a document discrimination means for discriminating the type of document based on the extracted character features using a trained machine learning model that outputs the type of document when character features are input, and a display data generation means for generating display data for displaying an image including the document image on a terminal device in order to receive information specifying the range of masking for specified items.

[0025] According to the present invention, it is possible to improve the accuracy of document sorting and to efficiently mask document images.

[0026] 1 is a schematic diagram showing an example of a schematic configuration of an information processing system according to an embodiment of the present invention. FIG. 1 is a schematic diagram showing an example of a document. FIG. 1 is a schematic diagram showing an example of a document. FIG. 1 is a schematic diagram showing an example of a document. FIG. 1 is a block diagram showing an example of a schematic configuration of the information processing server of FIG. 1. FIG. 1 is a schematic diagram showing an example of an image template. FIG. 1 is a schematic diagram showing an example of feature points of the image template. FIG. 1 is a flowchart showing an example of operation of the information processing server of FIG. 1. FIG. 1 is a schematic diagram showing an example of a masking reception screen. FIG. 1 is a schematic diagram showing an example of a masking reception screen. FIG. 1 is a flowchart showing a pattern matching sorting subroutine of FIG. 8A. FIG. 1 is a flowchart showing a subroutine of pattern matching sorting of the document of FIG. 9. FIG. 10 is a flowchart showing a code determination subroutine of FIG. 10. FIG. 10 is a flowchart showing a pattern matching determination subroutine of FIG. 10. FIG. 10 is a flowchart showing a feature point sorting subroutine of FIG. 8A. FIG. 13 is a flowchart showing a feature point sorting subroutine of the document of FIG. 13. FIG. 14 is a schematic diagram showing an example of a target document. FIG. 15 is a schematic diagram showing an example of feature points of the target document. FIG. 16 is a schematic diagram showing an example of matched feature points. FIG. 17 is a schematic diagram showing an example of unique and matched feature points. Fig. 19 is a flowchart showing a subroutine of AI sorting of Fig. 8A. Fig. 19 is a flowchart showing a subroutine of AI sorting of the document of Fig. 18. Fig. 20 is a schematic diagram showing an example of text data. Fig. 21 is a schematic diagram showing an example of the frequency of appearance of a divided character string. Fig. 22 is a schematic diagram showing an example of the frequency of appearance of a divided character string.

[0027] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the embodiment described below is an embodiment in which the present invention is applied to a management system.

[0028] [1. Overview of Configuration and Function of Management System] (1.1 Configuration and Function of Management System) First, the configuration and general functions of a management system according to one embodiment of the present invention will be described with reference to FIGS. 1 to 4. FIG.

[0029] Fig. 1 is a schematic diagram showing an example of the general configuration of an information processing system according to an embodiment of the present invention, Fig. 2A to Fig. 4B are schematic diagrams showing examples of documents.

[0030] 1, the information processing system 1 includes an information processing server device 10 that processes information about services such as applications, investigations, and account opening, a terminal device 20 for workers to work on documents, and a user terminal device 30 for users who apply for services. When accepting paper documents, the information processing system 1 may also include an image input device (not shown) that captures an image of the document and imports image data of the document image.

[0031] An information processing server device 10, which is an example of an information processing device, is a server of a BPO (Business Process Outsourcing) business, which is a service provider or service agent for applications, surveys, account opening, applications for various services, questionnaire surveys, etc. The information processing server device 10 performs information processing such as sorting document types, etc., based on image data of each document included in a set of documents received on paper or via the web.

[0032] The terminal device 20 is, for example, a mobile terminal such as a personal computer, a portable wireless telephone including a smartphone, or a tablet terminal. The terminal device 20 is installed according to each worker. The terminal device 20 displays an image of a document or the like on a display unit 21.

[0033] The user terminal device 30 is, for example, a mobile terminal such as a personal computer, a portable wireless telephone including a smartphone, a tablet terminal, etc. Using the user terminal device 30, a user can apply for services such as opening a bank account online, or download and print out application forms for services from the web.

[0034] The image input device includes a scanner or digital camera having an imaging element such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The image input device scans or captures documents received from the agent client or documents sent by the applicant, and generates image data. The image input device is connected to the information processing server device 10.

[0035] The information processing server device 10, the terminal device 20, and the user terminal device 30 are capable of transmitting and receiving data to and from each other via the network 3 using, for example, TCP / IP as a communication protocol. The network 3 is constructed, for example, by a local area network, the Internet, a dedicated communication line (for example, a CATV (Community Antenna Television) line), a mobile communication network, a gateway, or the like.

[0036] One example of a service provided by a BPO provider is an agency service that investigates and verifies the purpose of transactions by sending questionnaires or confirmation letters to customers who have opened accounts with financial institutions and already have transactions, depending on the transaction content and circumstances. This investigation is required because financial institutions are required to continuously verify customer information in order to strengthen measures against money laundering and terrorist financing. Customers must respond by mailing documents or accessing a website via email. Verification is required to be conducted periodically. For example, a BPO provider may print documents to be sent and mail them to customers or send them via communication methods such as email. Customers may handwrite the necessary information on documents such as questionnaires, seal them in an envelope, and send them to the financial institution along with a copy of their identification document or an image taken with a mobile device. Alternatively, the customer may enter the necessary information via a website and submit it. The BPO provider then receives the information mailed or entered via the website, extracts personal information from it, and conducts identity verification and other screening.

[0037] Here, we will provide an overview of the overall work flow of the services provided by BPO companies.

[0038] (1) Document Submission (Web-Based Acceptance) Survey subjects with bank accounts receive emails or envelopes. The emails or envelopes contain a URL for accessing a questionnaire to confirm the purpose of the transaction, etc., which users access and answer the questionnaire. A photograph of the identification document is taken using a smartphone camera or similar device, and the image of the identification document is also sent. The questionnaire is submitted as text data, and the identification document is submitted as image data. (2) Sorting (Determining Document Type) Images of the identification document are sorted by the information processing system 1 that implements the sorting system. As described below, different systems are used depending on the mix of document types, such as documents containing only standard documents, documents containing a mixture of standard documents and a small amount of non-standard documents, or a mixture of standard and non-standard documents. (3) Masking (Manual Masking on the Screen) Identification documents are masked to conceal sensitive information. Images of the identification documents are displayed on a terminal, and a worker manually masks sensitive information. Since this is done by a worker, there is no need to restrict whether the document is standard or non-standard. (4) Review In the information processing system 1, a worker compares the questionnaire text with the image of the identification document masked in (3) to confirm whether they are identical. (5) Delivery to customer In the information processing system 1, preparations are made to deliver the review results (match results with the identification document) and the questionnaire text data.

[0039] Next, the documents are questionnaires for confirming the purpose of the transaction, etc., applications for opening a bank account, identity verification documents, contracts, delivery notes, etc. A set of documents, which is an example of multiple related documents, is, for example, in the case of confirming the purpose of the transaction, etc., multiple documents such as questionnaires for confirming the purpose of the transaction, identity verification documents, etc. The set of documents may include a photograph of the person and multiple identity verification documents.

[0040] As shown in Figures 2A and 2B, the questionnaire (customer information confirmation document) has columns for each item, including name, gender, date of birth, address, telephone number, place of employment, occupation, purpose of transaction, assets, etc., and often consists of multiple pages, such as questionnaire document 40 on the first page and questionnaire document 41 on the second page.

[0041] Identification documents include driver's licenses, resident registration cards, family register cards, health insurance cards, passports, My Number cards, and receipts (such as receipts for utility bills such as electricity and gas, which include the person's name and address).

[0042] Examples of document types include questionnaires, bank account opening application forms (application forms, application forms), identity verification documents, blank sheets, and other documents. More specifically, questionnaires include the first, second, and third pages of a questionnaire, while identity verification documents include driver's licenses, passports, My Number cards, health insurance cards, resident registration cards, receipts, and so on.

[0043] Documents are also classified into standard documents, in which the positions of items written on the document are fixed for each document type, and non-standard documents, in which the positions of items written on the document are not fixed for each document type. Standard documents include driver's licenses, passports, and My Number cards. Non-standard documents include resident certificates, whose formats vary by municipality, and insurance cards and receipts, whose formats vary by issuer. As shown in FIG. 3 , the format of driver's license 42 is standardized nationwide, and driver's license 42 is classified as a standard document. On the other hand, as shown in FIGS. 4A and 4B , the format of resident certificates, like resident certificates 43 and 44, is not standardized, and therefore resident certificates are classified as non-standard documents.

[0044] Regarding a questionnaire document, if the location of each item on the questionnaire document is known in advance by the entity that determines the document type, the questionnaire document may be considered to be a standard document.

[0045] The information processing server device 10 also has a plurality of sorting engines, such as a sorting engine capable of sorting standard documents by document type, a sorting engine capable of sorting non-standard documents by document type, a sorting engine capable of sorting specific documents with high efficiency, etc. Here, a sorting engine is an algorithm or information processing method for determining the document type from image data of a document.

[0046] The above-mentioned review of task (4) involves comparing the application information in the application documents with the identity verification documents. Here, the review targets include review items such as the name and address written on the documents, as well as the entire document. The review items include, for example, review items that can be converted to text, such as the name in kanji, the name in kana, the address, and dates (date of birth, date of entry), as well as review items that are images, such as photographs and official seals. The application information is, for example, the content of each review item on the application documents.

[0047] There are various methods of review, such as exact match review, partial match review, exact content match review, date comparison review, and kana name review and judgment. Exact match review is a review that compares two review items to determine whether they match exactly, etc. Exact content match review is a review that compares a review item with a pre-set string to determine whether they match exactly, etc. Date comparison review is a review that compares the dates of two review items, or the date of a review item with a pre-set date to determine whether they match, etc. Kana name review is a review that determines whether a kana name matches a kanji name, etc.

[0048] In the screening process, screening questions are set, and the worker conducts the screening by answering the screening questions. There are various screening questions depending on the combination of the screening method, the document to be screened, and the screening items. For example, there are questions to check whether the name in kanji on the questionnaire document matches the name in kanji on the driver's license, which is the identification document, whether the address on the driver's license matches the address on the health insurance card, whether the name in kanji on the questionnaire document matches the name in kana, and whether the address on the questionnaire document matches the address on the resident registration card. Note that the screening questions may include information on answer options for the screening questions (such as "match," "mismatch," and "unreadable").

[0049] The screening process may be performed by a human worker, but may also involve automated screening using a computer. In automated screening, a computer performs screening by comparing the text information of the application documents and the identification documents in accordance with predetermined screening rules. In automated screening, the computer performs OCR on the identification documents to convert them into text, and then compares the text with the application information on the application documents to determine whether the two match. More specifically, for example, an exact match screening can be performed by converting the name written on the identification document into text using OCR and then comparing the text with the name text in the questionnaire using a computer. In this way, by using a computer to perform exact match screening using text strings, the screening process can be automated, thereby reducing the burden on the screening worker.

[0050] (1.2 Configuration and Function of Information Processing Server Device 10) Next, the configuration and function of the information processing server device 10 will be described with reference to Figs. 5 to 12.

[0051] Fig. 5 is a block diagram showing an example of a general configuration of the information processing server device 10. Fig. 6 is a schematic diagram showing an example of an image template. Fig. 7 is a schematic diagram showing an example of feature points of the image template.

[0052] As shown in Figure 5, the information processing server device 10, which is a computer, includes a control unit 11 that controls the information processing server device 10, a memory unit 12 that has various databases, a communication unit 13 that communicates with terminal devices 20, etc., and an output unit 14 that displays system management information, etc.

[0053] The control unit 11 has, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The control unit 11 may have a computing chip dedicated to AI calculations, such as a GPU (Graphics Processing Unit). The control unit 11 reads and executes various programs stored in the ROM or the memory unit 12 by the CPU, which reads and executes the various control programs stored in the ROM or the RAM. The control unit 11 controls each unit of the information processing server device 10 (the memory unit 12, the communication unit 13, the output unit 14, etc.). Note that the control unit 11 may read and execute these programs from a recording medium or the like that stores them.

[0054] The memory unit 12 (an example of a storage means) is configured with, for example, a hard disk drive, a silicon disk drive, etc. The memory unit 12 includes a template database 12a storing template images of each document, an image feature database 12b storing feature amounts of the template images of each document, an image database 12c storing image data of each document, and a business information database 12d storing associations between business information and applicable sorting engines. The memory unit 12 also stores various trained machine learning models for performing processes such as sorting using AI (artificial intelligence).

[0055] The template database 12a stores pixel data of blank template images in association with document type codes, etc., as shown in Fig. 6. Here, a template image is a sample image for identifying an image similar to a document.

[0056] Here, for example, the first two digits of the document type code may indicate a broad classification code such as "questionnaire document," "identification document," "blank sheet," or "other document." Furthermore, the lower digits of the document type code may indicate a more detailed classification such as "first page of questionnaire document," "second page of questionnaire document," or "third page of questionnaire document" for questionnaire documents, or a more detailed classification such as "driver's license," "passport," "My Number card," or "health insurance card" for identification documents. Furthermore, for resident certificates, which are non-standard documents, the template database 12a may not have template images. Templates may be categorized by business.

[0057] Furthermore, in cases where a one-dimensional code or two-dimensional code is displayed in advance on a questionnaire document or the like, information about the one-dimensional code or two-dimensional code to be affixed to the questionnaire document or the like is stored in the template database 12a in association with a code for the document type. Multiple one-dimensional codes or two-dimensional codes may be printed or otherwise set on a questionnaire document or the like. The total number of one-dimensional codes or two-dimensional codes is the number of set codes. In addition to the code indicating the document type, other codes such as information about the customer may also be included in the code.

[0058] Furthermore, the template database 12a stores image data of each one-dimensional code and two-dimensional code as a code template image, associated with the document type code. If the number of codes set is two or more, there is a template for each code image. This code image is an example of a template image.

[0059] Note that instead of one template image, multiple template images may be used for one document type. In this case, multiple pattern matching may be performed with the target document image using each template image. If two template images are used for one document type and pattern matching is performed twice, the number of pattern matching settings is "2." Multiple template images belonging to the same document type may be assigned a code obtained by adding a subnumber to the document type code. The number of pattern matching settings is stored in the template database 12a. The document type code and a code obtained by adding a subnumber or customer code to the document type code are examples of template information for template images for each document type. Furthermore, the template image may not be of the entire document, but may be of a portion of the document that is likely to have characteristics.

[0060] Furthermore, the template database 12a stores coordinate information for each item such as name, address, and gender, in association with a document type code, etc. Examples of coordinate information for an item include the range of the item field, and in the case of a name field, the coordinate information of the area where the name is written.

[0061] As shown in FIG. 7 , the image feature database 12b stores feature data of feature points p extracted by applying a feature extraction algorithm (e.g., Accelerated-KAZE, ORB (Oriented FAST and Rotated BRIEF), SIFT (Scale Invariant Feature Transform), etc.) to a template image in which each item is left blank, in association with a document type code, etc. Examples of feature point data include coordinate information of the feature points p and phase information of each feature point p. When determining feature points p, it is preferable to mask character information such as names and addresses and variable information in the template image in the same color as the background, or to remove white space around the form.

[0062] When determining feature point p, it is preferable to mask variable information in the template image, such as text information indicating items such as name and address, and areas indicating the location of the facial photograph, using the same color as the background, or to remove white space around the document. As mentioned above, variable information refers to information that differs for each individual identification document, such as name, address, and facial photograph. When feature point extraction is performed using a feature extraction algorithm, names and facial photographs may also be extracted as feature points, increasing the number of feature points to be matched, thereby increasing the amount of calculations and slowing down processing. Furthermore, because variable information differs for each identification document, it is difficult to use it as a feature for sorting or distortion correction (described later). Therefore, performing feature point matching using feature points extracted from variable information may slow down processing, which could result in reduced sorting accuracy. Removing variable information from the template image prevents slow feature point matching and reduced sorting accuracy. Feature point p in the template image is an example of a feature point extracted from an image in which variable information, including text information, has been removed from the template image.

[0063] The image database 12c stores the acquired image data in association with the document set ID of the document set, the document ID issued sequentially to each document, etc. The image database 12c also stores image data of items cut out from the document image in the process of dividing the image into individual document items in association with the document ID, item ID, etc.

[0064] The business information database 12d stores business information associated with customer information requesting the service, such as the opening of a bank account, service content such as questionnaire surveys, whether manual sorting is specified, information on the type of identification document to be used (such as driver's license only, or whether a resident registration card is also included), whether the document is standard or non-standard, and information on the method of submission (submission via the web, paper submission by mail, or a combination of these).

[0065] Furthermore, various trained machine learning models for performing processes such as sorting documents by type and detecting document items using AI are constructed in the storage unit 12. For example, trained machine learning models are constructed by inputting a feature vector in which character information of a learning document is used as one element into a plurality of machine learning models such as machine learning models for classification, such as a linear support vector machine (SVM) and a gradient boosting tree.

[0066] Furthermore, a confidence threshold of the machine learning model for classification is stored for each document type in the storage unit 12. This threshold is a judgment threshold for each document type that is determined in advance through testing, etc. For example, if the document type is a driver's license, the confidence threshold is xx%, and if the document type is an insurance card, the confidence threshold is x%, etc.

[0067] Furthermore, in the case of AI performing item detection processing, in the annotation, training data in which information tags are added to each item on a document is created, and machine learning is performed by the AI ​​to construct a machine learning model for item detection (an example of an item identification machine learning model). In particular, in the case of an AI model for masking items such as unnecessary personal information such as My Number and license conditions, and sensitive information such as medical history, in order to protect personal information, training data describing the masked areas (extraction of the item description area) is created in the annotation, and the AI ​​is trained to construct a machine learning model for masking. Note that machine learning may also be performed by mechanically detecting the difference between before and after the masking process from the actual document data. Furthermore, as an example of a machine learning model for item identification for each document type that is machine-learned for each document type so that the position of a specific item can be identified, machine learning may be performed for each document type so that each item is identified for each document type, and a machine learning model for item detection for each document type may be constructed in the memory unit 12.

[0068] The information may be managed by the same database of the same information processing server device 10, or each database may be stored in a database of a different information processing server device.

[0069] Various programs such as an operating system and a server program may also be stored in the storage unit 12. The various programs may be obtained, for example, from another server or the like via the network 3, or may be recorded on a recording medium and read via a drive device.

[0070] The communication unit 13 connects to the network 3 and controls communication with the terminal device 20 and the user terminal device 30 .

[0071] The output unit 14 is configured by, for example, a liquid crystal display element or an organic EL (Electro Luminescence) element.

[0072] 2. Operation example of information processing system 1 (Operation example of information processing server device) Next, an operation example of the information processing server device will be described with reference to Fig. 8A to Fig. 8C. Fig. 8A is a flowchart showing an operation example of the information processing server in Fig. 1. Fig. 8B and Fig. 8C are schematic diagrams showing an example of a masking reception screen.

[0073] First, in the above-mentioned operation (1), in the case of an online application, the information processing server device 10 acquires a series of image data of a set of documents, such as identification documents, from the user terminal device 30 and stores them in the storage unit 12 together with the document set ID of the document set and the document ID issued for each document. The identification documents are image data photographed with a mobile terminal device equipped with a camera. The set of documents may also include image data of the applicant's face.

[0074] In addition, in the case of questionnaire documents such as the questionnaire documents 40 and 41, the documents may be acquired as text data instead of image data. The information processing server device 10 acquires data from the user terminal device 30 that associates a document type ID, an item ID indicating each item, and text data entered in the field for each item displayed on the user terminal device 30.

[0075] Furthermore, questionnaire documents such as questionnaire documents 40 and 41 may be assigned a document ID such as a code in advance to identify each document. Examples of the assigned code include a one-dimensional code and a two-dimensional code. When applying online, the two-dimensional code or one-dimensional code may be displayed in advance in a predetermined position such as the bottom right of the document, or may be included in the image data later.

[0076] In the case of paper applications, the envelopes that have been mailed are opened. For example, the delivered cardboard box is opened, and an envelope containing a set of documents, such as a questionnaire containing necessary information and a copy of an identification document, is removed. The envelope is then opened, and the set of documents is removed. A document ID, such as a code, for identifying each document is assigned to each of the removed paper documents. An image input device scans each document in the set of documents, converts it into image data, and transmits it to the information processing server device 10. The information processing server device 10 acquires the series of image data of the set of documents from the image input device and stores it in the memory unit 12 together with the ID of the set of documents and the document IDs issued to each document.

[0077] 8A, the information processing server device 10 acquires business information (step S1). Specifically, the control unit 11 refers to the business information database 12d to acquire business information corresponding to the service of the outsourced customer.

[0078] Next, the information processing server device 10 checks the advance specification and determines whether manual sorting is specified (step S2). Specifically, the control unit 11 refers to the business information database 12d to check the advance specification of whether the document set will be sorted by document type using a sorting engine or manually, and determines whether manual sorting is required. Note that a host server device (not shown) that manages the entire system may exist for the information processing server device 10, and the host server device may check the advance specification of whether the document set will be sorted by document type using a sorting engine or manually.

[0079] If manual sorting is specified (step S2; YES), the information processing server device 10 registers manual sorting for the set of documents (step S3).

[0080] If there is no manual designation (step S2; NO), the information processing server device 10 determines whether or not the set of documents is only standard documents (step S4). Specifically, the control unit 11 determines whether or not the set of documents is only standard documents based on the business information.

[0081] If only a standard form is present (step S4; YES), the information processing server device 10 determines whether the image is distorted (step S5). Specifically, the control unit 11 calculates the degree of distortion of the shape from a rectangle through image processing. When a copy of an identification document or a photograph of an identification document is taken with a camera, the image may be distorted.

[0082] In this way, in the case of web submission, the applicant scans or photographs their identification document, etc., and sends a paper copy, so there is a possibility that the image may be distorted. Therefore, the information processing server device 10 determines that the image is distorted based on the business information and whether the document is submitted via the web. Note that instead of determining whether the image is distorted, the information processing server device 10 may determine whether the document contains a code, such as a two-dimensional code or a one-dimensional code. In this case, the presence of a code on the document corresponds to the presence of image distortion, and the absence of a code on the document corresponds to the absence of image distortion.

[0083] If the image is not distorted (step S5; NO), the information processing server device 10 performs pattern matching sorting (step S6). Specifically, the control unit 11 sorts each document by code determination, which reads the document code to determine the document type, or pattern matching (PM) determination, which determines the document type by pattern matching with a template image. The details of this process will be described later in the pattern matching sorting subroutine. Furthermore, in the case of web-based applications that contain only text data, pattern matching sorting may be omitted.

[0084] If the image is distorted (step S5; YES), the information processing server device 10 performs feature point sorting (step S7). Specifically, the control unit 11 sorts each document in the document set by pattern matching with the feature points of the template image. The details of this process will be described later in the feature point sorting subroutine.

[0085] If the documents are not of a fixed type only (step S4; NO), the information processing server device 10 performs AI sorting (step S8). Specifically, the control unit 11 extracts character features from text data generated by optically reading the documents, and sorts each document in the set into document types based on the extracted character features using a trained machine learning model that outputs document types when character features are input. The detailed processing content will be described later in the AI ​​sorting subroutine.

[0086] Next, after automatic sorting such as pattern matching sorting (step S6), feature point sorting (step S7), or AI sorting (step S8), or manual sorting (step S3), the information processing server device 10 displays masking work information (step S9). Specifically, as shown in FIG. 8B, the control unit 11 generates display data for displaying an image 50 including an image of the document on the display unit 21 of the terminal device 20 to accept information specifying the masking range for a specific item. To protect personal information, etc., masking work is performed on items such as unnecessary personal information such as My Number and license conditions, and sensitive information such as medical history. As shown in FIG. 8B, the control unit 11 may include information on the confirmed document type and information on the items to be masked in the display data of the image 50. Note that if the questionnaire document is text data, image data of the image 50 to be displayed on the terminal device 20 may be generated from the text data, the document type ID, and each item ID added to the text data. The control unit 11 may receive information specifying the range of masking for a predetermined item depending on the type of document.

[0087] In this way, it functions as an example of a display data generating means that generates display data for displaying an image including the document image on a terminal device in order to receive information specifying the masking range for a specific item.

[0088] Next, the information processing server device 10 transmits the displayed display data to the terminal device 20 of the worker in charge of the masking work. The terminal device 20 displays an image 50 of the document whose type has been identified, the document type, and a pointer 21p on a display unit 21 configured with a liquid crystal display element, an organic EL element, or the like. The worker operates the pointer 21p with a mouse or the like to specify the masking range for the portion of the item to be masked.

[0089] When the information specifying the masking range is received by the operator operating a pointer 21p using a mouse or the like, the terminal device 20 displays a semi-transparent mask 21a in which the area corresponding to the information specifying the range appears semi-transparent. As shown in Fig. 8B, the information in the "license conditions, etc." and the facial photograph are half-transparent, allowing the operator to recognize the area to be masked and the range of the mask.

[0090] In this way, when the masking range specification information is received, the display data generating means generates the display data in which the area corresponding to the range specification information is displayed semi-transparently.

[0091] Next, when the terminal device 20 receives the masking confirmation information by clicking the mouse, etc., it displays the area corresponding to the range specified by the information by filling in the area. As shown in Fig. 8C, the information in the "License Conditions, etc." section and the facial photograph are filled in black by the mask 50a.

[0092] In this way, when the masking confirmation information is received, the masking confirmation information functions as an example of a display data generating means that generates the display data for filling in the area corresponding to the range designation information.

[0093] Next, the terminal device 20 generates an image 51 from the image 50, in which descriptions of predetermined items are masked with a mask 50a, and transmits the image 51 to the information processing server device 10. The information processing server device 10 stores the image data of the image 51 in the image database 12c.

[0094] The display data for displaying an image including a document image on a terminal device to receive information on the masking range designation for a predetermined item, the display data for making the area corresponding to the masking range designation information appear translucent when the masking range designation information is received, and the display data for filling in the area corresponding to the masking range designation information when the masking confirmation information is received may be written in a markup language such as HTML (HyperText Markup Language). These display data are executed by the terminal device 20.

[0095] The processes of steps S1 to S5 may be performed by a human being. That is, a human being may determine which sorting engine to select from among pattern matching sorting, feature point sorting, and AI sorting.

[0096] In the example of an investigation into the purpose of transactions at financial institution accounts, the contents of a questionnaire are used to investigate whether fraudulent transactions have occurred, but identity verification using identification documents is required. By performing the sorting task (2), it is possible to confirm whether the appropriate identification documents are present and complete, enabling a reliable comparison of the identification documents and the questionnaire during the review. Because the masking task (3) is performed after the sorting task (2), masking efficiency is improved compared to manual tape masking. Because the information and locations to be masked differ for each type of identification document, separating documents by the same type allows for masking to be performed on documents of the same type, improving masking efficiency. Furthermore, by sorting the questionnaire documents in the sorting task (2), for example, by sorting them in the correct page order, delivery to the customer can be ensured. Furthermore, by sorting the identification documents in the sorting task (2), it is possible to reliably confirm whether the necessary identification documents are complete.

[0097] (2.1 Pattern Matching Sorting Subroutine) Next, the pattern matching sorting subroutine will be described with reference to Fig. 9. Fig. 9 shows the pattern matching sorting subroutine.

[0098] 9, the information processing server device 10 starts a loop process for the target image (an example of a document image) of each document in the document set (step S10). Specifically, the control unit 11 references the image database 12c to obtain the number of documents included in the document set, i.e., information on the target image, and performs processing in the loop in order from the document set according to the document ID. Note that if the questionnaire document is text data, it does not need to be included in the target image. The document type is determined by the document type ID added to the text data.

[0099] Next, the information processing server device 10 performs pattern matching sorting of the documents (step S11). Specifically, the control unit 11 determines the document type if there is only one candidate for determination using code determination, which reads the code on the document to determine the document type, or pattern matching (PM) determination, which determines the document type by pattern matching with a template image. If there is more than one candidate for determination, the document type is determined as undetermined, and the result of whether the document type is determined or undetermined is output. Note that the detailed processing content will be described later in the document pattern matching sorting subroutine.

[0100] In this way, the information processing server device 10 functions as an example of a standard document discrimination means that compares the template information with the document image information to discriminate the type of the document.

[0101] Next, the information processing server device 10 determines whether the document type has been determined (step S12). Specifically, the control unit 11 determines whether the document type has been determined based on the result of the document pattern matching sorting subroutine, which determines whether the document type has been determined or not. The control unit 11 refers to a flag indicating whether the document type has been determined or not.

[0102] If the document type is not determined (step S12; NO), the information processing server device 10 registers the document type as a target for manual sorting (step S13). Specifically, the control unit 11 stores the document ID of the target image in the database of the storage unit 12 as a target for manual sorting.

[0103] If the document type is determined (step S12; YES), the information processing server device 10 divides the image (step S14). Specifically, the control unit 11 refers to the template database 12a, acquires coordinate information for each item based on the document type ID of the document type, and cuts out the area corresponding to each item from the image data of the target image to generate an item image. The control unit 11 stores the item image data in the image database 12c in association with the document type ID of the document type and the item ID. Note that if the document type is "blank," the information processing server device 10 does not divide the image.

[0104] In this way, the information processing server device 10 functions as an example of a cutting means for cutting out an item image for an item according to the determined type of the document.

[0105] After image segmentation in step S14, or if the document type has not been determined (step S12; NO), the information processing server device 10 determines whether the number of loops has reached the number of target images. If the number of target images has not been reached, the information processing server device 10 returns to the processing in step S10 and proceeds to processing the next target image according to the next document ID. If the number of target images has been reached, the information processing server device 10 terminates the target image loop processing and terminates the pattern matching sorting subroutine (step S15). Note that the determination of document type sorting confirmation in step S12 and the manual sorting registration process in step S13 may be outside the target image loop. Furthermore, even if the document type is determined in manual sorting, the image segmentation process is also performed. After the type is determined in manual sorting, the loop processing returns to the processing in step S10, skipping step S11. Since the type has already been determined in step S12, the image segmentation process in step S14 is performed.

[0106] (2.1.1 Subroutine for sorting documents by pattern matching) Next, the subroutine for sorting documents by pattern matching will be described with reference to Fig. 10. Fig. 10 shows the subroutine for sorting documents by pattern matching.

[0107] 10, the information processing server device 10 acquires the target image (step S16). Specifically, the control unit 11 refers to the image database 12c and acquires image data of the target image according to the document ID of the document.

[0108] In this way, the information processing server device 10 functions as an example of a document image acquisition means that acquires document images of standardized documents in which the positions of items are determined for each document type.

[0109] Next, the information processing server device 10 performs a process for determining whether the target image is a blank page (step S17). Specifically, the control unit 11 calculates the area of ​​black parts in the image data of the target image, i.e., parts whose brightness is darker than a set threshold value.

[0110] Next, the information processing server device 10 determines whether the target image is a blank page (step S18). Specifically, if the ratio of the area of ​​black in the image data of the target image, i.e., the area of ​​parts with brightness darker than a set threshold, to the area of ​​the entire target image is less than the set threshold, the control unit 11 determines that the target image is a blank page.

[0111] Here, documents such as questionnaires are printed on only one side of a specified sheet of paper, and the back side may be blank. Furthermore, copies of identification documents are often made on one side of the sheet of paper. When scanning a document using an image input device, both sides of the document are scanned, taking into consideration the possibility that the document may have been printed or copied on both sides. If there is a side that has not been printed, filled in, or copied on, that image is sorted as a blank image and excluded from processing, thereby reducing the effort required to process and work with unnecessary images in post-processing and reducing the data volume.

[0112] If the page is not blank (step S18; NO), the information processing server device 10 performs code determination (step S19). Specifically, the control unit 11 reads the code value from the image of the one-dimensional code or two-dimensional code within the specified range by image processing, compares it with the codes in the template database 12a, and selects the document type corresponding to the matching code as a candidate. The details of this process will be described later in the code determination subroutine.

[0113] In this way, the information processing server device 10 functions as an example of a standard document discrimination means that compares the template information with the document image information to discriminate the type of the document.

[0114] Next, the information processing server device 10 performs a pattern matching determination (step S20). Specifically, the control unit 11 performs pattern matching between the target image and the template image on a pixel-by-pixel basis, calculates a score of the degree of match, and if the score is equal to or greater than a threshold, stores the result as a candidate document type. Note that the details of this process will be described later in the pattern matching determination subroutine. Note that the pattern matching determination may be performed before the code determination.

[0115] In this way, the information processing server device 10 functions as an example of a standard document discrimination means that compares the template information with the document image information to discriminate the type of the document.

[0116] Next, the information processing server device 10 determines whether the number of determination candidates is 1 or not (step S21). Specifically, the control unit 11 determines whether the number of determination candidates is 1 or not by code determination or pattern matching determination.

[0117] If the document is blank (step S18; YES), or if the number of determination candidates is 1 (step S21; YES), the information processing server device 10 determines that the document type has been determined (step S22). Specifically, the control unit 11 determines that the document type has been determined when the total number of determination candidates based on code determination and pattern matching determination is 1. For example, the control unit 11 sets a flag indicating that the document type has been determined, and stores the determined document type. Note that if the document is blank, the document type is set to "blank."

[0118] If the number of judgment candidates is not 1 (step S21; NO), for example, if there are 0 or 2 or more, the information processing server device 10 determines that the document type is undetermined (step S23). Specifically, if the control unit 11 determines that the total number of judgment candidates based on the code judgment and pattern matching judgment is not 1, the document type is undetermined. For example, the control unit 11 sets a flag indicating that the document type is undetermined. If the results of the code judgment and the PM judgment are different, two results will be registered for one document. In this case, the two judgments do not match, so the document type is undetermined. Even if only one of the code judgment or the pattern matching judgment is available, it is determined as long as it is uniquely determined.

[0119] (2.1.1.1 Chord Determination Subroutine) Next, the chord determination subroutine will be described with reference to Fig. 11. Fig. 11 shows the chord determination subroutine of Fig. 10.

[0120] As shown in FIG. 11 , the information processing server device 10 acquires a code from the target image (step S25). Specifically, the control unit 11 converts an image of a one-dimensional code or two-dimensional code within a predetermined range into a code value based on coordinate information of the code in the document. If there are multiple codes in a single document, each code is read and the code in the target image is acquired. When acquiring the codes in the target image, the information processing server device 10 counts the number of times a code in the target image has been acquired and calculates the number of target codes. At this time, the control unit 11 may cut out a code image of the portion where the code is located from the target image. If there are multiple codes, each corresponding code image is cut out.

[0121] In this way, the information processing server device 10 functions as an example of a code reading unit that reads the code written on the document from the document image.

[0122] Next, the information processing server device 10 starts processing the image type loop (step S26). Specifically, the control unit 11 refers to the image database 12c and performs the number of image types and the image type loop processing. Note that the codes of the template images to be matched may be narrowed down based on the business information.

[0123] Next, the information processing server device 10 acquires the set codes (step S27). Specifically, the control unit 11 refers to the template database 12a and acquires the set one-dimensional code or two-dimensional code corresponding to the template image. When acquiring the set codes, the information processing server device 10 counts the number of codes to be read and calculates the number of set codes. Note that the information processing server device 10 may also acquire the number of set codes by referring to the template database 12a.

[0124] In this way, the information processing server device 10 functions as an example of a template information acquisition unit that acquires template information of template images prepared for each type of document.

[0125] Next, the information processing server device 10 starts processing the loop of the set code (step S28). Specifically, the control unit 11 performs the processing in the loop of the set code the number of times equal to the set code number, starting from the document set.

[0126] Next, the information processing server device 10 starts processing the loop of the code in the target image (step S29). Specifically, the control unit 11 performs processing in the loop of the code in the target image a number of times equal to the number of target codes.

[0127] Next, the information processing server device 10 compares the code values ​​(step S30). Specifically, the control unit 11 compares the code value of the target image with the code value of the template image. Note that if the set code includes a code related to a customer, the information processing server device 10 extracts only the document type code and compares it with the code of the template image.

[0128] In this way, the information processing server device 10 functions as an example of a standard document discrimination means that compares the read code with the code associated with the template image to determine the type of the document.

[0129] Next, the information processing server device 10 determines whether the code values ​​match (step S31).

[0130] If the code values ​​match (step S31; YES), the information processing server device 10 adds the code values ​​to the candidates (step S32). Specifically, the control unit 11 stores the matched code values ​​and counts the number of candidates for determination.

[0131] After step S32, or if the code values ​​do not match (step S31; NO), the information processing server device 10 determines whether the number of loops has reached the number of codes in the target image. If the number of codes has not been reached, the information processing server device 10 returns to the processing of step S29 and compares the code values ​​with the code in the next target image. If the number of codes in the target image has been reached, the loop processing of the codes in the target image ends (step S33).

[0132] Next, the information processing server device 10 determines whether the number of loops has reached the set number of codes, and if the set number of codes has not been reached, returns to the processing of step S28 and starts a new loop of the codes in the target image, whereas if the set number of codes has been reached, ends the loop processing of the set codes (step S34).

[0133] Next, the information processing server device 10 determines whether the number of loops has reached the number of image types. If the number of image types has not been reached, the information processing server device 10 returns to the processing in step S26 and starts a new loop for the set code. If the number of image types has been reached, the image type loop processing ends and the code determination subroutine ends (step S35).

[0134] (2.1.1.2 Subroutine for Determining Pattern Matching) Next, the subroutine for determining pattern matching will be described with reference to FIG.

[0135] FIG. 12 shows the pattern matching determination subroutine of FIG.

[0136] As shown in FIG. 12, the information processing server device 10 starts the loop process of the image type as in step S26 (step S36).

[0137] Next, the information processing server device 10 acquires the set template image (step S37). Specifically, the control unit 11 refers to the template database 12a based on the document type code and acquires a template image corresponding to the document type code. If the number of pattern matching settings is not 1, template images equal to the number of pattern matching settings are acquired according to the document type code with branch numbers. The information processing server device 10 acquires the number of pattern matching settings by referring to the template database 12a. The template image may be a template image of a code, a template image of the entire document, or a template image of a portion of a document.

[0138] Next, the information processing server device 10 starts the set pattern matching loop (step S38). Specifically, the control unit 11 performs the process in the set pattern matching loop the number of times equal to the set number of pattern matchings.

[0139] Next, the information processing server device 10 performs pattern matching (step S39). Specifically, the control unit 11 performs pattern matching between the target image and the read-out template image on a pixel-by-pixel basis, and obtains a score (likelihood) of the likelihood of the pattern. For example, a convolution operation is performed between the target image and the template image to calculate the maximum value, and a score is calculated from this maximum value as the degree of match. In the case of a code image, matching is performed between the code image 41a cut out from the target image and the template image of the code image.

[0140] In this way, the information processing server device 10 functions as an example of a standard document discrimination means that compares the document image with the template image and discriminates the type of the document.

[0141] Next, the information processing server device 10 determines whether the score is equal to or greater than a threshold value (step S40). For example, if the score is equal to or greater than a predetermined value, it is determined that the template matches.

[0142] If the score is equal to or greater than the threshold (step S40; YES), the information processing server device 10 adds the template image to the candidates (step S41). Specifically, the control unit 11 stores the document type code of the matching template image and the score.

[0143] If the score is not equal to or greater than the threshold (step S40; NO), the information processing server device 10 skips step S41 of adding the candidate.

[0144] Next, the information processing server device 10 determines whether the number of loops has reached the set number of pattern matching, and if the set number of pattern matching has not been reached, returns to the processing of step S38 and performs pattern matching, etc. using the template image of the next branch number. If the set number of pattern matching has been reached, the set pattern matching loop processing ends (step S42).

[0145] Next, the information processing server device 10 determines whether the number of loops has reached the number of image types, and if not, returns to the processing of step S36 and starts a new pattern matching loop that has been set. If the number of image types has been reached, the information processing server device 10 ends the loop processing of the image type loop and ends the pattern matching determination subroutine (step S43).

[0146] (2.2 Feature Point Classification Subroutine) Next, the feature point classification subroutine will be described with reference to Fig. 13. Fig. 13 shows the feature point classification subroutine of Fig. 8.

[0147] 13, the information processing server device 10 starts a loop process for the target image of each document in the document set (step S50), similar to the process in step S10. Note that if the questionnaire document is text data, it does not need to be included in the target image. The document type is determined by the document type ID added to the text data.

[0148] Next, the information processing server device 10 sorts the document by its feature points (step S51). Specifically, the control unit 11 compares the document with the feature points of the template image, counts the number of unique and matching feature points, and determines the document type if the count is equal to or greater than a predetermined number. If the count is less than the predetermined number, the document type is left undetermined, and outputs the result of whether the document type is determined or undetermined. The details of this process will be described later in the document feature point sorting subroutine.

[0149] In this way, the information processing server device 10 functions as an example of a standard document discrimination means that compares the template information with the document image information to discriminate the type of the document.

[0150] Next, the information processing server device 10 determines whether the document type has been determined (step S52). Specifically, the control unit 11 determines whether the document type has been determined based on the result of the document feature point sorting subroutine, which determines whether the document type has been determined or not.

[0151] If the document type is not determined (step S52; NO), the information processing server device 10 registers the document type as a target for manual sorting (step S53). Specifically, the control unit 11 stores the document ID of the target image in the database of the storage unit 12 as a target for manual sorting.

[0152] If the type of document is determined (step S52; YES), the information processing server device 10 divides the image (step S54), as in step S14.

[0153] In this way, the information processing server device 10 functions as an example of a cutting means for cutting out an item image for an item according to the determined type of the document.

[0154] Next, the information processing server device 10 determines whether the number of loops has reached the number of target images, and if the number of target images has not been reached, returns to the processing of step S50 and proceeds to processing for the next target image according to the next document ID, and if the number of target images has been reached, ends the target image loop processing and ends the feature point sorting subroutine (step S55). Note that the determination of document type sorting confirmation in step S52 and the processing of manual sorting registration in step S53 may be outside the target image loop.

[0155] (2.2.1 Subroutine for sorting feature points of document) Next, the subroutine for sorting feature points of document will be described with reference to Fig. 14. Fig. 14 shows the subroutine for sorting feature points of document in Fig. 13 .

[0156] 14, the information processing server device 10 acquires feature points of the template images (step S56). Specifically, the control unit 11 refers to the image feature amount database 12b to acquire information on the feature points of each template image.

[0157] In this way, the information processing server device 10 functions as an example of a template information acquisition means that acquires template information of template images prepared for each document type. The information processing server device 10 also functions as an example of a template information acquisition means that acquires information on feature points of the template images.

[0158] Next, the information processing server device 10 acquires the target image (step S57), as in step S16. For example, the control unit 11 acquires the target image 45 as shown in Fig. 15. In this way, the information processing server device 10 functions as an example of a document image acquisition means that acquires a document image of a standard document in which the positions of items are determined for each document type.

[0159] Next, the information processing server device 10 detects feature points of the target image (step S58). Specifically, the control unit 11 applies a feature extraction algorithm to the image data of the target image to extract feature points p of the target image, as shown in Fig. 16. For example, intersections and corners of ruled lines are feature points.

[0160] Next, the information processing server device 10 counts the number of matched unique feature points (step S59). Specifically, the control unit 11 refers to the image feature database 12b, compares a group of feature points p of the target image (as shown in FIG. 16) with a group of feature points of the template image (as shown in FIG. 7), calculates the Hamming distance, and extracts the closest feature point Mp as the matched feature point. For example, as shown in FIG. 17A, in the case of template image #99-12345 in FIG. 7, feature points Mp corresponding to the feature points of the square shape are extracted from the target image, and in the case of template image #99-12346 in FIG. 7, feature points Mp corresponding to the feature points of the lattice square shape are extracted from the target image. Note that when extracting matched feature points, the first and second closest feature points are also extracted and used to select unique feature points, as described below.

[0161] In this way, the information processing server device 10 functions as an example of a standard document discrimination means that extracts the feature points from the document image, compares the extracted feature points with the feature points of the template image, and determines the type of the document.

[0162] Next, the control unit 11 detects unique feature points Up (an example of selected feature points) from among the matched feature points Mp (corresponding feature points). Among the feature points matched with the feature points of the template image, the most similar and second most similar feature points are extracted from the target image. If the similarity rate is equal to or greater than a threshold, the most similar feature point is selected as a unique feature point, i.e., a matched unique feature point. That is, as shown in FIG. 17B , if the most similar feature point and the second most similar feature point are similar to each other by a certain percentage or more, the matched feature point is deemed to be a unique feature point but not a unique feature point, and is removed from the valid feature point list. This allows feature points that are unlikely to be mismatched to be detected as matched unique feature points. For example, as shown in FIG. 17B , a grid feature point is removed because a similar feature point exists next to it, leaving only feature points with no other similar feature points.

[0163] Here, we will explain how to determine whether feature points are similar. Accelerated-KAZE features are represented as bit strings (binary values) of 0s and 1s. Therefore, when two bit strings are aligned, the number of digits by which 0s and 1s differ (the Hamming distance) can be used to compare similarities. As shown below, feature points are compared using the Hamming distance when selecting feature points. If the Hamming distance A between a feature point in the template image and its closest feature point in the target image is less than or equal to the Hamming distance B between the feature point in the template image and its second-closest feature point in the target image, the closest feature point is counted as a matching unique feature point. For example, if the Hamming distance A is less than or equal to the Hamming distance B × 0.8, the feature point is counted as a matching unique feature point.

[0164] This makes it possible to select a "unique feature point" that has no similar feature points other than the most similar feature point in the target image, preventing matching errors where similar feature points are mistakenly matched even for target images that have been rotated or distorted during shooting.

[0165] In this way, the information processing server device 10 functions as an example of a standard document discrimination means that discriminates the type of the document based on the number of the selected feature points and the number of feature points of the template image.

[0166] Next, the information processing server device 10 calculates the similarity for each template image (step S60). Specifically, the control unit 11 calculates the similarity of "number of matched unique feature points / number of feature points in the template image" and records it as the similarity for the template image. For example, as shown in FIGS. 7 and 17B, for the feature points of template image #99-12345, the number of feature points in the template image is "18," the number of matched unique feature points is "4," and the similarity for the template images is 4 / 18 = 0.222. For the feature points of template image #99-12346, the number of feature points in the template image is "16," the number of matched unique feature points is "4," and the similarity for the template images is 4 / 16 = 0.25.

[0167] Next, the information processing server device 10 determines a template image of the sorting result (step S61). Specifically, the control unit 11 determines the template image with the highest similarity among the similarities calculated in step S60 as the sorting result.

[0168] In this way, the information processing server device 10 functions as an example of a standard document discrimination means that discriminates the type of the document based on the number of the selected feature points and the number of feature points of the template image.

[0169] Next, the information processing server device 10 determines whether the number of matched unique feature points is equal to or greater than a threshold (step S62). Specifically, the control unit 11 determines whether the number of matched unique feature points in the template image with the highest similarity is equal to or greater than a predetermined value. Even if the template image has the highest similarity, if the number of matched unique feature points in the target image is less than the predetermined value, there is a possibility of erroneous determination or inability to properly correct distortion, and therefore this process is performed.

[0170] In this way, the information processing server device 10 functions as an example of a standard document discrimination means that discriminates the type of the document based on the number of feature points of the target image that correspond to the feature points of the template image.

[0171] If the number of matched feature points is equal to or greater than the threshold (step S62; YES), the information processing server device 10 calculates a homography transformation matrix (step S63). Specifically, the control unit 11 calculates a homography transformation matrix using each of the extracted, matched unique feature points, and applies the homography transformation matrix to calculate a distortion-corrected image of the target image.

[0172] In this way, the information processing server device 10 functions as an example of an image correcting means that corrects the document image based on the feature points extracted from the document image.

[0173] Next, the information processing server device 10 determines whether the converted area is within a normal range (step S64). Specifically, the control unit 11 multiplies the area calculated from the four corners of the target image (image before distortion correction) and each of the four corner points of the target image by a homography transformation matrix to detect "each point corresponding to the four corners of the target image" in the coordinate system of the distortion-corrected image, and compares it with the converted area calculated from these points to determine whether the converted area is within a normal range.

[0174] If the converted area is within the normal range (step S64; YES), the information processing server device 10 outputs the sorting result (step S65). Specifically, the control unit 11 stores the confirmed document type and sets a flag indicating that the document type is confirmed.

[0175] Next, the information processing server device 10 outputs the image after the distortion correction (step S66). Specifically, the control unit 11 transmits the image after the distortion correction, in which the document type has been determined, to the terminal device 20 for the next task.

[0176] If the number of matched feature points is less than the threshold value (step S62; NO), or if the converted area is not within the normal range (step S64; NO), the information processing server device 10 determines that the document type is undetermined (step S67). Specifically, the control unit 11 sets a flag indicating that the document type is undetermined.

[0177] (2.3 AI Sorting Subroutine) Next, the feature point sorting subroutine will be described with reference to Fig. 18. Fig. 18 shows the AI ​​sorting subroutine of Fig. 8.

[0178] 18, the information processing server device 10 starts a loop process for the target image of each document in the document set (step S70), similar to the process in step S10. Note that if the questionnaire document is text data, it does not need to be included in the target image. The document type is determined by the document type ID added to the text data.

[0179] Next, the information processing server device 10 performs AI sorting of the documents (step S71). Specifically, the control unit 11 determines the document type using AI from the character features of the target image, and outputs the result of whether the document type is confirmed or unconfirmed. Note that the details of the process will be described later in the subroutine for AI document sorting.

[0180] If the document type is not determined (step S72; NO), the information processing server device 10 registers the document type as a target for manual sorting (step S73). Specifically, the control unit 11 stores the document ID of the target image in the database of the storage unit 12 as a target for manual sorting.

[0181] If the document type is confirmed (step S72; YES), the information processing server device 10 determines whether the number of loops has reached the number of target images, and if the number of target images has not been reached, returns to the processing of step S70 and proceeds to processing for the next target image according to the next document ID, and if the number of target images has been reached, ends the target image loop processing and ends the AI ​​sorting subroutine (step S74). Note that the determination of document type sorting confirmation in step S72 and the processing of manual sorting registration in step S73 may be outside the target image loop.

[0182] After the document type is determined in step S72, the position of a predetermined item may be specified in the document image according to the determined document type, and an item image for the specified item may be extracted.

[0183] Specifically, the control unit 11 refers to the storage unit 12, selects a machine learning model for detecting corresponding items based on the confirmed document type, and identifies the description area of ​​each item for the document whose document type has been confirmed using the machine learning for the selected document type. The control unit 11 cuts out the area corresponding to each item from the coordinate information of each item whose description area has been identified and the image data of the target image, and generates an item image. The control unit 11 stores the item image data in the image database 12c in association with the document type ID and item ID of the document type.

[0184] In this way, the information processing server device 10 functions as an example of an extraction means that identifies the position of a specified item in the document image according to the determined type of the document, and extracts an item image for the identified item.

[0185] (2.3.1 Subroutine for AI document sorting) Next, the subroutine for AI document sorting will be described with reference to Fig. 19. Fig. 19 shows the subroutine for AI document sorting of Fig. 18.

[0186] 19, the information processing server device 10 acquires a target image (step S75) as in step S16. For example, the control unit 11 acquires image data of a document such as a questionnaire document 40 as shown in FIG. 2A.

[0187] In this way, the information processing server device 10 functions as an example of a document image acquisition means that acquires document images of documents including personal identification documents.

[0188] Next, the information processing server device 10 performs a sharpening process (step S76). Specifically, the control unit 11 applies a filter that enhances the contrast of edges to the image data to enhance the contours of the target image.

[0189] Next, the information processing server device 10 performs binarization processing (step S77). Specifically, the control unit 11 converts the luminance value of a pixel to "1" if the luminance value is equal to or greater than a predetermined luminance value, and converts the luminance value of the pixel to "0" if the luminance value is less than the predetermined luminance value. By performing binarization and sharpening processing, noise is removed and character contours are made more prominent for the next OCR processing.

[0190] Next, the information processing server device 10 performs OCR processing (step S78). Specifically, the control unit 11 performs OCR processing on the image data that has been sharpened and binarized, and concatenates multiple pieces of text data obtained to generate text data such as that shown in Fig. 20. In the case of horizontal writing, the control unit 11 scans the image horizontally from the top left and concatenates the text data in order.

[0191] Next, the information processing server device 10 determines the document type using AI (step S79). Specifically, the control unit 11 divides the character string into one-character units and two-character units, which are examples of predetermined character units. Character n-grams can be used to divide text data. Character n-grams are a method of dividing a document into n consecutive characters. When dividing a character string into two-character units, the text data in FIG. 20 is divided into two consecutive characters. As shown in FIG. 21A, the data is divided into two-character units, such as "O-Kan-Yaku-Sama" (customer), "Kan-Sama" (customer), etc. When dividing a character string into one-character units, the data is divided into "O-Kan-Yaku-Sama" (customer), "Kan-Sama" (customer), etc., as shown in FIG. 21B. Examples of character strings divided into multiple character units include strings divided into 1-character units and 2-character units, strings divided into 1-character units and 3-character units, strings divided into 2-character units and 3-character units, and strings divided into 1-character units, 2-character units and 3-character units. However, since the text is converted using OCR, there is a possibility of characters being misread or omitted, so it is preferable that single-character units are included.

[0192] Next, the control unit 11 divides one piece of text data into a plurality of character strings of a predetermined length, and then calculates the frequency of occurrence of each character string within the text data. As shown in FIG. 21A, the frequency of occurrence is calculated for each character string divided into two characters. As shown in FIG. 21B, the frequency of occurrence is calculated for each character string divided into one character. Two feature vectors are extracted, each of which has each divided character string as an element and the frequency of occurrence of that character string as its element value. That is, as shown in FIG. 21A, a feature vector is composed of two-character character strings and the frequency of occurrence of the character string, and as shown in FIG. 21B, a feature vector is composed of one-character character strings and the frequency of occurrence of the character string. These feature vectors are also referred to as character features. The number of character strings divided into two-character units and the number of character strings divided into one-character units constitute the number of dimensions of the feature vector.

[0193] Furthermore, the character strings divided by the control unit 11 may be weighted according to their importance. For example, a weighting method such as the TF-IDF method is used. Two other feature vectors (feature vectors with weighted elements of appearance frequency) are extracted, each of which has an element corresponding to each divided character string and an element value weighted by the appearance frequency of the character string.

[0194] Here, TF (Term Frequency) represents the frequency of occurrence of a certain character string in a document. Here, "in document" refers to all the text data collected to determine the weighting. The more frequently a character string appears in document data, the more important the character string is considered to be. IDF (Inverse Document Frequency) represents the inverse of the document frequency in which a certain character string appears. A character string that appears in many texts is considered to have low importance within a document. Ultimately, the TF-IDF value is calculated by multiplying the TF value by the IDF value. Therefore, even if a character string appears frequently in a document (= has a high TF value), if that character string appears in many documents, its importance will be low. For example, on a character-by-character basis, characters such as "month" and "day" are considered to appear frequently, but since they are present in many identification documents, their TF-IDF value will be low and they will be considered to have low importance when sorting documents. Furthermore, a certain character string that appears frequently in a document but appears less frequently in other identification documents will have a high TF-IDF value.

[0195] Therefore, by using a learning model that has learned which character strings appear with high TF-IDF values ​​in a particular identification document, the control unit 11 can perform the sorting process. For example, in a driver's license, which is an identification document, the character string "driver" is unlikely to appear in other identification documents such as My Number and insurance cards, and is therefore thought to have a high TF-IDF value. Therefore, in determining the document type using this AI, when an identification document image is read and the character string "driver" appears in the image, the control unit 11 can determine that the document type of the identification document image is a "driver's license."

[0196] In this way, the information processing server device 10 functions as an example of a character feature extraction means that extracts character features from text data generated by optically reading the document image. The information processing server device 10 extracts the character features configured by character strings obtained by dividing the text data into predetermined character units and the frequency of appearance of each of the divided character strings in the text data, and functions as an example of a character feature extraction means having a plurality of the predetermined character units. The information processing server device 10 functions as an example of a character feature extraction means that extracts the character features by weighting the frequency of appearance of each of the divided character strings.

[0197] Next, the control unit 11 inputs each feature vector for the target image in two-character units and single-character units into multiple machine learning models for classification, outputs the confidence level for each document type, and outputs the maximum value as the AI ​​sorting result.

[0198] Next, the information processing server device 10 determines whether the confidence level for each document type is equal to or greater than the threshold value for that document type (step S80). Specifically, the control unit 11 refers to the database in the storage unit 12 and determines whether the confidence level for the document type with the maximum value is equal to or greater than the threshold value for that document type.

[0199] In this way, the information processing server device 10 functions as an example of a document discrimination means that uses a trained machine learning model for document type that outputs a document type when character features are input, and discriminates the document type based on the extracted character features.

[0200] If the number is equal to or greater than the threshold (step S80; YES), the information processing server device 10 outputs the document classification results (step S81). Specifically, the control unit 11 stores the confirmed document classification and sets a flag indicating that the document classification has been confirmed.

[0201] If the difference is not equal to or greater than the threshold value (step S80; NO), the information processing server device 10 determines that the document type is undetermined (step S82). Specifically, the control unit 11 sets a flag indicating that the document type is undetermined.

[0202] In this way, by performing tasks (1) to (3), documents are collected via online reception in task (1), and sorted (determined by document type) by the information processing system 1 in task (2). Also, manual masking is performed on the screen in task (3), and sensitive information is masked.

[0203] The sorted identification documents and questionnaire documents are then examined in operation (4). In this examination, in the information processing system 1, the worker compares the extracted image (extracted name) with the image of the identification document on the work screen to confirm whether they are identical. In sorting operation (2), the questionnaire documents and identification documents are sorted; for example, documents on which a name is written on the questionnaire are identified in sorting operation (2). In addition, in the same sorting operation (2), the identification documents are sorted and the type of identification document is determined.

[0204] In order to strengthen measures against money laundering and terrorist financing, financial institutions and other institutions are required by guidelines and other regulations to verify the identity of the customer using identification documents when investigating account transactions. In this embodiment, by performing steps (1) to (3), it is possible to easily compare the questionnaire documents with the required identification documents, thereby enabling the review of step (4) to be carried out more quickly and reliably. If there are any issues, such as the customer not having submitted the appropriate identification documents, there being insufficient identification documents, or the name on the questionnaire documents not matching the name on the identification documents (resulting in a mismatch), the documents are returned to the customer and they are prompted to resubmit the missing documents, etc.

[0205] When investigating account transactions at financial institutions, the required identification documents and types are determined by the case. For example, if the identification document has a photograph (such as a driver's license or My Number card), one document is required, and if the identification document does not have a photograph (such as a health insurance card or resident registration card), two documents are required. Incorrect or insufficient identification documents refer to situations where an incorrect identification document is submitted, or where only one identification document is submitted when two types should be submitted. By sorting in operation (2) and identifying the type of identification document, it is possible to reliably confirm whether the appropriate identification document has been submitted. The review results (match results with the identification documents) and the text data of the questionnaire in the information processing system 1 are prepared for delivery to the customer in operation (5). The contents of the text questionnaire are then reviewed by the customer's bank, etc., and ultimately checked for any fraudulent use of the account.

[0206] According to this embodiment, a document image of a standard document in which the positions of items are determined according to the document type is acquired, template information of a template image prepared for each document type is acquired, the template information is compared with the document image information to determine the document type, and display data is generated for displaying an image including the document image on a terminal device to accept information specifying the masking range for the specified items.By specializing this for sorting standard document types, the template information can be effectively used to improve the accuracy of document sorting.

[0207] Furthermore, after determining the type of document, display data for an image 50 for displaying an image including the document image on a terminal device is generated to accept information specifying the masking range for a specific item. The display data is then sent to the worker's terminal device 20, and the image 50 is displayed on the display unit 21 of the terminal device 20. The masking operation, in which a rectangle is drawn around areas corresponding to personal information, can be performed efficiently through on-screen operations. While the masking area varies depending on the type of identification document, the relevant personal information can be shielded. Furthermore, as shown in FIG. 8B , displaying information about the items to be masked reduces the risk of erroneous masking and makes masking easier for the worker.

[0208] Furthermore, when information specifying a masking range is received, display data is generated in which the area corresponding to the information specifying the range is displayed semi-transparently, and when information confirming the masking is received, display data is generated in which the area corresponding to the information specifying the range is filled in. Since the area is semi-transparent immediately after being surrounded by the pointer 21p or the like, it is easy for the worker to see the description to be masked and the masked area. By performing the registration or confirmation process, the description of the item to be masked is filled in by the mask 50a and becomes invisible.

[0209] In addition, since the data can be handled digitally from the start without requiring paper output, it is easier to automate subsequent processes. In the case of online reception, the data arrives as text data for the questionnaire and image data for the identity verification document, so all that is required is to send the image data to the sorting system, eliminating the need to open envelopes when they are sent and the process of scanning paper documents, improving work efficiency.

[0210] When information on feature points p of a template image is obtained, feature points p are extracted from a document image, and the extracted feature points p are compared with the feature points p of the template image to determine the type of document, the feature points can be used to quickly and accurately sort documents.

[0211] Among the feature points p extracted from the document image, feature points Mp corresponding to feature points p of the template image are extracted, and among the corresponding feature points Mp, feature points are selected according to the similarity between the feature points, such as the Hamming distance, and when determining the document type based on the number of the selected feature points Up, it is possible to select a unique feature point Up that has no similar feature points other than the most similar feature point in the target image for sorting the document type, thereby preventing matching errors where similar feature points are mistakenly matched even for target images that have been rotated or distorted during photography.

[0212] When determining the type of document based on the number of selected feature points Up and the number of feature points p of the template image, the greater the number of selected feature points Up relative to the number of feature points p of the template image, the more similar the target image can be determined to be to the template image, thereby further improving the accuracy of document sorting.

[0213] A user may send a photograph of their identification document taken with a camera on a mobile device or the like. Because the user places their identification document on a desk or other surface to take the photograph, the image of the identification document may be distorted, for example, at an angle, depending on the position of the mobile device used to take the photograph. As a result, the document image may be sent in a distorted state. The distortion of the distorted document image is corrected based on the feature points Up, and OCR is performed on the corrected document image, improving the accuracy of the OCR. In particular, when performing automated review in task (4), OCR is performed on the image of the identification document to convert the items listed on the identification document into text. Therefore, correcting the distortion of the document image improves the accuracy of the OCR, which ultimately improves the accuracy of the automated review.

[0214] When determining the type of document based on the number of feature points of the target image that correspond to the feature points of the template image, if the number of matching feature points Mp is too small, the document can be determined as NG, which serves as a measure to prevent incorrect sorting and further improve the accuracy of document sorting.

[0215] When a document image is corrected based on the feature points Up selected in the document image, distortion is corrected based on the feature points Up, so that the accuracy of image correction can be improved.

[0216] When a document image is compared with a template image to determine the type of document, pattern matching can improve the accuracy of document sorting.

[0217] When a code written on a document is read from a document image and the read code is compared with a code associated with a template image to determine the type of document, the code can be used to quickly determine the type of document. In addition, by combining the code value read from the code with pattern matching of the code image, the accuracy of document sorting can be further improved.

[0218] In addition, by sorting and identifying the document type and then cutting out the item image for each item, the cut-out part is determined depending on the document type, so the image can be cut out efficiently. Since personal information such as names can be cut out and separated, it is possible to separate workers who handle each item, which leads to the protection of information.

[0219] Document images of documents including identification documents are obtained, character features are extracted from text data generated by optically reading the document images, and a trained machine learning model is used to output the document type when the character features are input, thereby determining the document type based on the extracted character features, and generating display data for displaying an image including the document image on a terminal device to accept information on the masking range designation for specified items.By converting the entire image into text using OCR and generating character features, it is possible to handle not only standard documents such as driver's licenses and residence cards, but also non-standard documents such as insurance cards and resident registration cards that are difficult to classify using image processing, and further improving the accuracy of document sorting.

[0220] The text data is divided into character strings by a predetermined number of characters, and character features are extracted based on the frequency of appearance of each character string in the text data. If there are multiple character strings, the accuracy of document sorting can be further improved by using the multiple character features. For example, by acquiring two types of features, one for each character and one for each two characters, significant features can be obtained even if the document is in poor condition and the text information as a result of OCR is noisy.

[0221] When at least one predetermined character unit is a string of two or more characters, if a character is missing due to blurring of the character or the like, a different character string is likely to be generated and other components will appear in the feature vector, but when at least one predetermined character unit is a single character unit, only a part of the component of the feature vector is missing, so the feature vector does not change significantly and there is less impact on the accuracy of document sorting.

[0222] When extracting character features with weights applied to the frequency of occurrence of each divided character string, weighting the frequency of occurrence using TF-IDF or the like makes it possible to weight a feature that appears frequently in data obtained by OCR conversion of one form into text, but appears less frequently in data obtained by OCR conversion of another form, and therefore this feature is likely to be important. Therefore, by using character features with this weighting, the accuracy of document sorting can be further improved.

[0223] It should be noted that multiple machine learning models may be used when determining the type of document using AI. For example, combining multiple classifiers such as linear SVM and gradient boosted trees can improve the accuracy of document classification.

[0224] 10: Information processing server device (information processing device) 20: Terminal device 40, 41, 42, 43, 44: Document 50, 51: Image 50b: Masking

Claims

1. An information processing device comprising: a document image acquisition means for acquiring a document image of a standardized document in which the positions of items are determined according to the type of document; a template information acquisition means for acquiring template information of a template image prepared for each of the document types; a standardized document discrimination means for comparing the template information with information on the document image to discriminate the type of the document; and a display data generation means for generating display data for displaying an image including the document image on a terminal device for receiving information specifying the masking range for a predetermined of the items.

2. An information processing device according to claim 1, characterized in that, when the display data generation means receives information specifying the masking range, it generates the display data in which the area corresponding to the information specifying the range is displayed semi-transparently, and, when it receives information confirming the masking, it generates the display data in which the area corresponding to the information specifying the range is filled in.

3. An information processing device according to claim 1, wherein the template information acquisition means acquires information on feature points of the template image, and the standard document discrimination means extracts feature points from the document image and compares the extracted feature points with the feature points of the template image to discriminate the type of the document.

4. An information processing apparatus according to claim 3, wherein the feature points of the template image are feature points extracted from an image obtained by removing variable information including character information from the template image.

5. An information processing device according to claim 3 or 4, characterized in that the standard document discrimination means extracts feature points corresponding to feature points of the template image from the feature points extracted from the document image, selects feature points from the corresponding feature points according to the Hamming distance between the feature points, and discriminates the type of the document based on the number of selected feature points.

6. An information processing apparatus according to claim 5, wherein said standard document discrimination means discriminates the type of said document based on the number of said selected feature points and the number of feature points of said template image.

7. An information processing apparatus according to claim 3, characterized in that the type of the document is determined based on the number of feature points of the document image corresponding to the feature points of the template image.

8. An information processing apparatus according to claim 3, further comprising image correction means for correcting the document image based on feature points extracted from the document image.

9. An information processing device according to claim 1, further comprising a code reading means for reading a code printed on the document from the document image, wherein the standard document discrimination means compares the read code with a code associated with the template image to discriminate the type of the document.

10. An information processing apparatus according to claim 1 or 9, wherein said standard document discrimination means discriminates the type of said document by comparing said document image with said template image.

11. An information processing device comprising: a document image acquisition means for acquiring a document image of a document including an identification document; a character feature extraction means for extracting character features from text data generated by optically reading the document image; a document discrimination means for discriminating the type of the document based on the extracted character features using a trained machine learning model that outputs a document type when character features are input; and a display data generation means for generating display data for displaying an image including the document image on a terminal device in order to receive information specifying the masking range for a specified item.

12. An information processing device according to claim 11, characterized in that, when the display data generation means receives information specifying a range for the masking, it generates the display data in which the area corresponding to the information specifying the range is displayed semi-transparently, and, when it receives information confirming the masking, it generates the display data in which the area corresponding to the information specifying the range is filled in.

13. An information processing device according to claim 11 or 12, wherein the character feature extraction means extracts the character feature consisting of character strings obtained by dividing the text data into predetermined character units and the frequency of occurrence of each of the divided character strings in the text data, and wherein there are a plurality of the predetermined character units.

14. An information processing apparatus according to claim 13, wherein at least one of the predetermined character units is a one-character unit.

15. An information processing apparatus according to claim 13, wherein said character feature extraction means extracts said character feature by weighting the frequency of occurrence of each of said divided character strings.

16. An information processing method comprising: a document image acquiring step in which a document image acquiring means acquires a document image of a standardized document in which the positions of items are determined according to the type of document; a template information acquiring step in which a template information acquiring means acquires template information of a template image prepared for each type of document; a standardized document determining step in which a standardized document discrimination means compares the template information with information about the document image and then discriminates the type of the document; and a display data generating step in which a display data generating means generates display data for displaying an image including the document image on a terminal device for receiving information specifying the range of masking for a predetermined one of the items.

17. An information processing method comprising: a document image acquisition step in which a document image acquisition means acquires a document image of a document including an identification document; a character feature extraction step in which a character feature extraction means extracts character features from text data generated by optically reading the document image; a document discrimination step in which a document discrimination means discriminates the type of the document based on the extracted character features using a trained machine learning model that outputs a document type when character features are input; and a display data generation step in which a display data generation means generates display data for displaying an image including the document image on a terminal device in order to accept information specifying the masking range for a specified item.

18. A program for an information processing device, characterized in that it causes a computer to function as: document image acquisition means for acquiring a document image of a standardized document in which the positions of items are determined according to the type of document; template information acquisition means for acquiring template information of a template image prepared for each type of document; standardized document discrimination means for comparing the template information with information on the document image to discriminate the type of document; and display data generation means for receiving information specifying the masking range for a specified one of the items and generating display data for displaying an image including the document image on a terminal device.

19. A program for an information processing device, comprising: a document image acquisition means for acquiring document images of documents including identification documents; a character feature extraction means for extracting character features from text data generated by optically reading the document images; a document discrimination means for discriminating the type of the document based on the extracted character features using a trained machine learning model that outputs a document type when character features are input; and a display data generation means for generating display data for displaying an image including an image of the document on a terminal device in order to accept information specifying the masking range for specified items.

Images