Information processing device, method for controlling the information processing device, and program

Abstract

This reduces the burden on users during the masking area verification process. [Solution] The information processing device for performing string masking includes: extraction means for extracting strings from an input document image; selection means for selecting attribute types to be masked from among the attribute types of strings contained in the document image based on user instructions; acquisition means for obtaining the confidence level that each string contained in the document image is of the selected attribute type; processing means for performing masking on the region of the document image in which the confidence level is above a threshold; and display control means for displaying a preview image of the masked document image on a display means.

Description

Technical Field

[0001] The present disclosure relates to the extraction of masking targets when masking images.

Background Art

[0002] When sharing document images such as identity documents, application documents, and drawings containing personal information or confidential information with others, a part of the document may be filled in for masking (hereinafter also referred to as "blacking out" or "masking"). As a masking method, there are a method of masking an area having a predetermined position and size in a document image, and a method of automatically extracting and masking an area to be masked in which personal information or confidential information is described in the document image each time. In the method of automatically extracting and masking the area to be masked, if the accuracy of the automatic extraction decreases, there may be a case where not all the areas to be masked can be extracted and masked. In order to avoid such a situation, for example, in Patent Document 1, a method is disclosed in which a masking area is displayed on a preview screen so that the user can confirm whether there is an excess, a deficiency, or an error in the masking area.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] <00000 twenty-five ]] In the technique disclosed in Patent Document 1, it is necessary for the user to individually confirm whether there is an excess or deficiency in the masking area on the preview screen, and the confirmation work becomes troublesome when the number of confirmation targets increases. The present disclosure has been made in view of this point, and an object thereof is to reduce the burden on the user in the confirmation work of the masking area.

Means for Solving the Problems

[0005] The information processing device according to this disclosure is an information processing device that performs string masking processing, and is characterized by comprising: extraction means for extracting strings from an input document image; selection means for selecting, based on user instructions, the attribute types to be targeted for the masking processing from among the attribute types of strings contained in the document image; acquisition means for obtaining the confidence level that each string contained in the document image is of the selected attribute type; processing means for performing the masking processing on the region of the document image in which the confidence level is equal to or greater than a threshold; and display control means for displaying a preview image of the masked document image on a display means. [Effects of the Invention]

[0006] The technology disclosed herein makes it possible to reduce the burden on users in the process of verifying masked areas. [Brief explanation of the drawing]

[0007] [Figure 1] A diagram showing an example of the configuration of an information processing system. [Figure 2] A diagram showing an example of the hardware configuration of a processing terminal that constitutes an information processing system. [Figure 3] A diagram showing an example of the hardware configuration of a processing terminal that constitutes an information processing system. [Figure 4] A diagram showing an example of the software configuration of a processing terminal that constitutes an information processing system. [Figure 5] A flowchart illustrating the entire process performed by an information processing system. [Figure 6] A diagram showing an example of a scan screen in an information processing system. [Figure 7] A diagram showing an example of corrected image data resulting from image correction processing in an information processing system. [Figure 8] A diagram showing an example of a preview screen for an information processing system. [Figure 9] A flowchart illustrating the process of determining what to mask in an information processing system. [Figure 10]A diagram illustrating the process of switching the masking target in an information processing system. [Figure 11] A flowchart illustrating the entire process performed by an information processing system. [Figure 12] A flowchart illustrating the ranking process in an information processing system. [Figure 13] A flowchart illustrating the process of determining what to mask in an information processing system. [Figure 14] A diagram illustrating the process of switching the masking target in an information processing system. [Figure 15] This diagram illustrates an example where the extracted string has two or more attribute types. [Figure 16] This diagram illustrates an example where the extracted string has two or more attribute types. [Modes for carrying out the invention]

[0008] The embodiments of this disclosure will be described below with reference to the attached drawings. The embodiments described below are not limiting to this disclosure, and not all combinations of features described in the embodiments are necessarily essential to the solutions of this disclosure. The same components will be denoted by the same reference numerals. Furthermore, each step in the flowchart will be indicated by a reference numeral beginning with "S".

[0009] [First Embodiment] 《Overall Structure》 Figure 1 shows the overall configuration of the information processing system according to this embodiment. The information processing system in Figure 1 includes an MFP (MultiFunction Peripheral) 110 and external storage 120. The MFP 110 is connected to a server that provides various services on the Internet via a LAN (Local Area Network).

[0010] The MFP110 is a multifunction peripheral having multiple functions such as a scanner and a printer, and is an example of the information processing apparatus according to the present disclosure. The MFP110 also has a function of transferring a scanned image file to a service that can save the file in an external storage service or the like. Note that the information processing apparatus of the present disclosure is not limited to a multifunction peripheral having a scanner and a printer, and may be a personal computer (PC), a tablet terminal, or a smartphone.

[0011] The external storage (service) 120 is a service that can save a file received via the Internet and can acquire the file from an external device via a web browser. The external storage 120 is, for example, a cloud service. The external storage is not limited to only the external storage 120, and a plurality of external storages may exist.

[0012] The information processing system according to the present embodiment has a configuration including the MFP110 and the external storage 120, but is not limited thereto. For example, part of the functions and processes of the MFP110 may be implemented on another server arranged on the Internet or on a LAN. Also, the external storage 120 may be arranged on a LAN instead of on the Internet. Further, the external storage 120 may be replaced with a mail server or the like, and a scanned image may be attached to and transmitted by mail. The MFP110 may also have the saving function of the external storage 120.

[0013] 《Hardware Configuration of MFP》 FIG. 2 is a diagram showing an example of the hardware configuration of the MFP 110. The MFP 110 includes a control unit 210, an operation unit 220, a printer 221, a scanner 222, and a modem 223. The control unit 210 includes a CPU 211, a ROM 212, a RAM 213, and a HDD (Hard Disc Drive) 214. Further, the control unit 210 includes an operation unit I / F 215, a printer I / F 216, a scanner I / F 217, a modem I / F 218, and a network I / F 219, and controls the operation of the entire MFP 110. The CPU 211 reads out a control program stored in the ROM 212 or the HDD 214, executes various functions of the MFP 110 such as scanning, printing, and communication, and controls the MFP 110. The RAM 213 is used as a main memory of the CPU 211 and a temporary storage area such as a work area. In the present embodiment, it is assumed that one CPU 211 executes each process shown in the flowchart below using one memory (RAM 213 or HDD 214), but the present invention is not limited to this. For example, a plurality of CPUs and a plurality of RAMs or HDDs may cooperate to execute each process. The HDD 214 is a large-capacity storage unit that stores image data and various programs. The HDD 214 may be an SSD (Solid State Drive), a flash memory, or cloud storage.

[0014] The operation unit I / F 215 is an interface connecting the operation unit 220 and the control unit 210. The operation unit 220 includes a touch panel or keyboard, and accepts user operations, inputs, and instructions. By using a touch panel, the operation unit 220 also functions as a reception unit and display unit for user operations. The printer I / F 216 is an interface connecting the printer 221 and the control unit 210. Image data for printing is transferred from the control unit 210 to the printer 221 via the printer I / F 216 and printed on the printing medium. The scanner I / F 217 is an interface connecting the scanner 222 and the control unit 210. The scanner 222 reads a document set on a document glass (not shown) or ADF (Auto Document Feeder) to generate image data and inputs it to the control unit 210 via the scanner I / F 217. The MFP 110 can print (copy) the image data generated by the scanner 222 from the printer 221, as well as send it as a file or via email. The MFP110 can also display scanned image data on an operation unit 220, such as a touch panel. The modem I / F 218 is an interface that connects the modem 223 and the control unit 210. The modem 223 transmits image data via facsimile to a facsimile machine on the PSTN. The network I / F 219 is an interface that connects the control unit 210 (MFP110) to a LAN. The MFP110 uses the network I / F 219 to transmit image data and information to various services on the Internet and to receive various information. The LAN may be a wired LAN or a wireless LAN.

[0015] External storage hardware configuration Figure 3 shows an example of the hardware configuration of the external storage 120. The external storage 120 includes a CPU 311, ROM 312, RAM 313, HDD 314, and network I / F 315. The CPU 311 controls the operation of the entire external storage 120 by reading control programs stored in the ROM 312 and executing various processes. The RAM 313 is used as the main memory of the CPU 311 and as a temporary storage area such as a work area. The HDD 314 is a large-capacity storage unit that stores image data or various programs. The HDD 314 may be an SSD or flash memory. The HDD 314 may also be cloud storage. The network I / F 315 is an interface that connects the external storage 120 to the internet. The external storage 120 receives processing requests from other devices such as the MFP 110 via the network I / F 315 and sends and receives various information.

[0016] 《MFP Software Configuration》 Figure 4 shows an example of the software configuration of the information processing system according to this embodiment. The MFP110 is broadly divided into two parts: the native function unit 410 and the additional function unit 420. Each function unit is realized when the CPU 211 reads a program stored in the ROM 212 or HDD 214 of the MFP110 into the RAM 213 and executes it. Each part included in the native function unit 410 is standard equipment of the MFP110. The additional function unit 420 is an application that is additionally installed on the MFP110. The additional function unit 420 is a Java®-based application, and it is possible to easily add functions to the MFP110. Note that other additional applications not shown may be installed on the MFP110.

[0017] The native function unit 410 includes a scan execution unit 411, an internal data storage unit 412, a print execution unit 413, and a user interface (UI) display unit 414. The additional function unit 420 includes a main processing unit 421, a scan instruction unit 422, an image processing unit 423, a data management unit 424, a print instruction unit 425, and an internet access unit 426. The additional function unit 420 also includes a display control unit 427, a document type determination unit 428, an attribute detection unit 429, a confidence level acquisition unit 430, a masking target determination unit 431, and a selection unit 432.

[0018] The main processing unit 421 has a control function for the overall processing of the additional function unit 420. Specifically, the main processing unit 421 controls the entire processing of the additional function unit 420 and requests processing from each part included in the additional function unit 420.

[0019] The scan instruction unit 422 requests the scan execution unit 411 to perform a scan according to the scan settings entered via the user interface screen (UI screen). The scan execution unit 411 receives the scan request from the scan instruction unit 422, including the scan settings. In accordance with the scan request, the scan execution unit 411 generates scanned image data by scanning the document placed on the document glass with the scanner 222 via the scanner I / F 217. The generated scanned image data is sent to the internal data storage unit 412. The scan execution unit 411 sends an image identifier that uniquely identifies the stored scanned image data to the scan instruction unit 422. The image identifier consists of numbers, symbols, and letters to uniquely identify the scanned image data in the MFP 110. The internal data storage unit 412 stores the scanned image data received from the scan execution unit 411 in the HDD 214.

[0020] The image processing unit 423 performs analysis and processing on image data, such as scanned image data. Regarding scanned image data, the image processing unit 423 receives an image identifier from the scan instruction unit 422 and retrieves the scanned image data corresponding to the image identifier from the internal data storage unit 412. The image processing unit 423 performs image processing such as character area analysis, OCR (Optical Character Recognition) processing, image rotation, and image tilt correction within the retrieved scanned image data (document image). Furthermore, the image processing unit 423 synthesizes a mask image corresponding to the masking area and instructs the internal data storage unit 412 to save the generated masking image data. A masking area is information representing the coordinates of the start and end points of a rectangular area to be masked within an image. For example, a masking area is information representing the coordinates of the start and end points of a rectangular area such as "(441,957), (1369,1057)". In this embodiment, the masking area is detected from the OCR results by the attribute detection unit 429, and is determined after confirmation and modification by the user via the display control unit 427. In this embodiment, the masking area is a rectangular area, but the masking area may be any shape such as an ellipse or a triangle. A masked image is an image in which the masking area is masked on the image data. Masking may be done by filling with black, filling with the background color, or by any other method. The image processing unit 423 transmits the image identifier to the print instruction unit 425 and the internet access unit 426 according to the set output settings. Therefore, the image processing unit 423 includes a string extraction unit 433 that extracts strings by OCR processing and a masking processing unit 434 that performs masking processing. The data management unit 424 associates information such as the file name and save location of the masked image with the image identifier and saves it as history in the HDD 214.

[0021] The print instruction unit 425 transmits a print processing request corresponding to the print settings entered via the UI screen and an image identifier received from the image processing unit 423 to the print execution unit 413. The print execution unit 413 receives the print request, including the print settings, and the image identifier from the print instruction unit 425. The print execution unit 413 retrieves image data corresponding to the image identifier from the internal data storage unit 412 and generates printable image data according to the print request. The print execution unit 413 prints an image, such as a masked image, on the printing medium using the printer 221 via the printer I / F 216, according to the generated printable image data.

[0022] The Internet access unit 426 sends processing requests to cloud services that provide storage functions (storage services). Cloud services generally use protocols such as REST or SOAP and expose various interfaces for saving files to cloud storage and retrieving saved files from external devices. The Internet access unit 426 operates the cloud service using the exposed interfaces of the cloud service. The Internet access unit 426 obtains the file and transmission information corresponding to the image identifier received from the image processing unit 423 from the data management unit 424. Using the transmission information obtained from the data management unit 424, the Internet access unit 426 transmits the file obtained from the data management unit 424 to the external storage 120 via the network interface.

[0023] The display control unit 427 displays a UI screen on the touch panel-enabled liquid crystal display unit of the MFP110's operation unit 220 to accept user operations. The UI screen includes operation screens that accept operations such as scan settings, scan start operation, scanned image preview, operation on masking areas (described later), masking image preview, list printing, output settings, or output start operation. The selection unit 432 selects predetermined items from the UI screen displayed by the display control unit 427. For example, the selection unit 432 selects the attribute type to be masked from among all attribute types using a UI screen (selection screen) displayed for user operation (see Figure 6).

[0024] The document type determination unit 428 infers the document type of the document image from the string. The document type represents the purpose of the document, such as "contract," "delivery note," or "invoice." In this embodiment, the document type is determined using machine learning, which is trained on the features of multiple document types using sample documents of those types as training data. In this embodiment, a Transformer-based machine learning model is used. A bidirectional Long Short-Term Memory (LSTM) neural network, a Sequence to Sequence model, or a recurrent neural network (RNN) may also be used for document type determination.

[0025] The attribute detection unit 429 detects attribute types representing information attributes such as personal information or confidential information from the string received from the image processing unit 423 and requests the data management unit 424 to save them to the HDD 214. An attribute type is the type of concept represented by a string and is used to specify the string to be masked (masking string). In this embodiment, detectable attribute types represent, but are not limited to, types of personal information or confidential information such as "name," "credit card number," and "email address." The information that can be obtained as a result of the attribute detection process is the attribute type and the string corresponding to the attribute type. For example, "Yamada Taro" is obtained as the string corresponding to the attribute type "name." Note that there is not limited to one string corresponding to one attribute type. If there are multiple names listed in the document, for example, multiple strings such as "Yamada Taro" and "Yamada Hanako" may be obtained as strings corresponding to the attribute type "name."

[0026] The attribute detection unit 429 may switch processing using the document type, which is the processing result of the document type determination unit 428. For example, the attribute type detection engine may be switched based on the document type. Alternatively, for example, the attribute type "Name" may be output as "Contractor" if the document type is "Contract," and as "Person in Charge" if the document type is "Invoice," etc., converted from the detection result of a uniform attribute type detection engine based on the document type. In this embodiment, the attribute detection unit 429 holds items of a predetermined detectable attribute type, but the location where these items are held is not limited to the attribute detection unit 429.

[0027] The confidence level acquisition unit 430 acquires a confidence level for each attribute detection result detected by the attribute detection unit 429. The confidence level is a value that indicates the certainty of the attribute detection result. For example, if the string "Yamada Taro" is detected as attribute type "Name", the confidence level indicates the certainty that it is indeed correct as attribute type "Name". In other words, the confidence level is a measure that represents the certainty of the attribute extracted as each attribute type. The confidence level may be acquired only for the attribute type selected by the selection unit on the UI screen described above, or it may be acquired for all attribute types. The masking target determination unit 431 ultimately determines the area to be masked on the document image based on the user's instructions acquired via the display control unit 427 and the confidence level acquired by the confidence level acquisition unit 430.

[0028] 《Overall Processing Flow》 The processes described below are realized when the CPU 211 of the MFP110 reads a control program stored in the ROM 212 or HDD 214 and executes and controls the various functions of the MFP110 and the functions of additional applications.

[0029] Figure 5 is a flowchart for generating a masking image from an image scanned by the MFP 110 and printing it. In addition to printing, the scanned image may also be saved as a file and sent to external storage 120 such as cloud storage. In this embodiment, an example is described in which the display control unit 427 displays the screen on the touch panel of the operation unit 220, but the display control unit 427 may also provide each screen in this embodiment to another device, and the operation unit or display unit of the other device may display each screen. Furthermore, a configuration may be made in which a part of the processing provided by the additional function unit 420 is performed by an external processing unit.

[0030] An additional application (hereinafter referred to as the "masking app") that masks a portion of a scanned image and sends it to a cloud service becomes available when installed on the MFP110. Once the masking app is installed on the MFP110, a button to use the application's functions will appear on the MFP110's main screen.

[0031] In S501, the main processing unit 421 requests the display control unit 427 to display the scan screen of the masking application. The display control unit 427 generates the scan screen and displays it on the touch panel of the operation unit 220 via the UI display unit 414. Through the scan screen, the main processing unit 421 receives instructions from the user regarding masking settings representing the attribute type to be masked and the execution of the scan.

[0032] Figure 6 shows an example of the scan screen in this embodiment. The scan screen 600 includes a message 601, a select all checkbox 602, an attribute type specification checkbox 603, and a scan button 604. Message 601 displays a message prompting the user to take action. In the example in Figure 6, an example of a message prompting the selection and scanning of the masking target is shown. The select all checkbox 602 is a checkbox that selects or deselects all checkboxes in the attribute type specification checkbox 603.

[0033] The attribute type specification checkbox 603 is a checkbox for specifying the attribute of the area to be masked. When the user presses the attribute type specification checkbox 603, the attribute is selected or deselected as the attribute to be masked. The attribute type specification checkbox 603 displays the attribute type according to the attribute that can be detected by the attribute detection unit 429. Figure 6 shows an example in which "Name," "Postal Code," "Address," and "Amount" are selected as the attribute types to be masked by the user operation in the selection unit 432 described above. In this embodiment, as an example of a scan screen, Figure 6 shows an example of a screen that accepts the specification of attribute types, but it is not limited to this. The system may accept the document type from the user, display the attribute types associated with the document type, and then accept the attribute type, or it may accept only the document type from the user and automatically determine the attribute types associated with the document type without displaying them. The scan button 604 is a button that confirms the attribute type specified in the attribute type specification checkbox 603 as the masking setting and starts scanning. When the scan button 604 is pressed by the user, the process proceeds to S502 in Figure 5.

[0034] Returning to the explanation of Figure 5, in S502, the main processing unit 421 requests a scan from the scan execution unit 411 via the scan instruction unit 422 and acquires image data (document image). In this embodiment, the main processing unit 421 acquires image data by scanning, but it may also acquire image data from the HDD 214 via the internal data storage unit 412. The main processing unit 421 may also acquire image data from the external storage 120 via the internet access unit 426. Alternatively, image data may be acquired using any method.

[0035] In S503, the main processing unit 421 requests the image processing unit 423 to perform skew rotation correction on the image data acquired in S502. The image processing unit 423 corrects the skew rotation of the image data and generates corrected image data. Figure 7 shows an example of the corrected image processing in this embodiment. Figure 7(a) is the image data before correction, showing an example where the characters in the document are tilted due to skew during scanning. Figure 7(b) is the image data after skew rotation correction, showing an example where the tilt of the characters is corrected. In addition, skew rotation correction may be performed when the image is upside down or rotated by 90 degrees. In this way, skew rotation correction can correct the image to one suitable for the Block Selection (BS) / OCR processing described later. Here, Block Selection (BS) refers to the process of extracting blocks of each content type from the target image. Content types include text (characters), images, and graphics. OCR processing is performed on blocks that are determined to be text (characters).

[0036] In S504, the main processing unit 421 requests the image processing unit 423 to perform BS / OCR processing on the corrected image data generated in S503. The image processing unit 423 performs BS / OCR processing on the corrected image data and extracts text and text regions. Table 1 shows an example of text and text regions in the BS / OCR results of this embodiment.

[0037] [Table 1]

[0038] In S505, the main processing unit 421 requests the attribute detection unit 429 to detect the attribute to be masked, using the masking settings acquired in S501 and the string and string area extracted in S504. The attribute detection unit 429 performs attribute detection processing from the string and stores the attribute type and the string corresponding to the attribute type in RAM 213. As a method of attribute detection, for example, rule-based attribute detection may be used. Rule-based attribute detection is a method of detecting attributes based on predefined rules, such as the position in the document, font size, string format, or relative position to a key string corresponding to a specific keyword. For example, for attribute type "Title", a large-font-size string belonging to the top of the document is extracted. For attribute type "Date", strings such as "Date" or "Issue Date" are used as key strings, and strings such as "2019-04-01" that match the yyyy-mm-dd format are extracted based on that. The attribute type "Document Number" extracts strings such as "1234" based on key strings such as "Document Number," "No.", or "#". The attribute type "Postal Code" extracts strings such as "213-0039" or "123-0045" that match the ddd-dddd format. Note that the extraction method for attribute detection processing is not limited to rule-based attribute detection. For example, attribute detection may be performed using a pre-trained machine learning model. As a machine learning model, a Transformer-based model can be used. Alternatively, a bidirectional Long Short-Term Memory (LSTM) neural network, Sequence to Sequence model, or Recurrent Neural Network (RNN) may be used.

[0039] As a result of attribute detection, for example, a string representing a company name such as "Iroha Co., Ltd." is estimated to have the attribute type "Company Name". An example of the attribute type of a string estimated in this embodiment is shown in Table 2.

[0040] [Table 2]

[0041] Note that the accuracy of attribute type estimation in the attribute detection unit 429 is not 100%, and the estimation results may contain errors. In the attribute type estimation example in Table 2, the string "123-0045," which is actually a string indicating the document number of the invoice, is incorrectly estimated as the attribute type "postal code." Also, not all strings extracted in S504 are assigned an attribute type. For example, the string "To Whom It May Concern" and the string "Invoice Amount," which corresponds to the key string, may not be assigned an attribute type.

[0042] Next, the attribute detection unit 429 uses the string area corresponding to the attribute type specified in the masking setting in S501 as the masking item. In other words, the string corresponding to the attribute type selected by the selection unit 432 becomes the masking item. For example, in the attribute type specification checkbox 603 in Figure 6, "Name," "Postal Code," "Address," and "Amount" are specified. As an example of a masking item, Table 3 shows an example of an item to be masked in this embodiment.

[0043] [Table 3]

[0044] In S506, the main processing unit 421 requests the confidence acquisition unit 430 to acquire the confidence level for each masking item detected in S505. In this embodiment, the confidence acquisition unit 430 acquires the confidence level output by the attribute detection unit 429 along with the attribute detection result. The confidence level is a value that indicates the likelihood of the attribute detection result. For example, if the string "Kiyano" is detected as attribute type "Name", the confidence level indicates the likelihood that it is indeed correct as attribute type "Name". When using rule-based attribute detection, it is possible to calculate the confidence level, which indicates the likelihood according to the degree of match with the rule. For example, a higher confidence level is output the closer the distance to the key string, and a lower confidence level is output the further away the distance to the key string. Another possible method is to output a high confidence level when a key string is detected and a string that matches the format is also detected, but output a low confidence level when a key string is not detected and a string that matches the format is detected. Alternatively, a score is calculated indicating whether the extracted string matches the format, whether the extracted string is positioned in the designated location, and a score indicating the recognition rate of the extracted OCR result. The confidence score may then be calculated by multiplying the calculated scores together or by a weighted average of the calculated scores. As an alternative, if a pre-trained machine learning model is used for attribute detection, its output confidence score may be used.

[0045] The method for obtaining the confidence score is not limited to obtaining the confidence score output by the attribute detection unit 429. For example, if the BS / OCR results performed by the image processing unit 423 include the confidence score for each string or character, that may be used. Alternatively, the confidence score acquisition unit 430 may newly estimate the confidence score. For example, it may be calculated using a predetermined string format for each attribute type of masking item, and the degree of agreement between the string and its string format.

[0046] Furthermore, although this embodiment obtains confidence levels only for masking items, it is not limited to this. The confidence level acquisition unit 430 may acquire confidence levels for all attribute types selected on the UI screen for each string. As an example of confidence levels, Table 4 shows an example of confidence levels acquired in this embodiment.

[0047] [Table 4]

[0048] In this embodiment, the confidence level is expressed as a real number between 0.0 and 1.0, with a higher value indicating a more reliable attribute detection result obtained in S505. The confidence level may also be expressed as a number between 0% and 100%. Specifically, the confidence level of the string "213-0039," which was correctly estimated as the attribute type "postal code," is higher than the confidence level of the string "123-0045," which was incorrectly estimated as the attribute type "postal code" and actually represents a document number. The confidence level does not need to be a continuous value; for example, it may be determined by rule matching / mismatch, or a continuous value may be divided by a threshold, resulting in discrete values ​​such as two levels ("high" and "low") or three levels ("high," "medium," and "low").

[0049] In S507, the main processing unit 421 requests the display control unit 427 to generate a preview screen using the corrected image data generated in S503 and the masking items and confidence levels extracted in S505 and S506. The display control unit 427 generates a preview screen in which rectangles corresponding to the coordinate positions of the masking items are superimposed on the corrected image data as masking areas, and displays it on the touch panel of the operation unit 220 via the UI display unit 414. On the preview screen, the user can confirm and correct the masking areas. Details of the preview screen will be described later. Once the user confirms the masking areas on the preview screen, the main processing unit 421 proceeds to S508.

[0050] In S508, the main processing unit 421 requests the image processing unit 423 to generate a masking image using the masking region determined in S507 and the corrected image data generated in S503. The image processing unit 423 performs masking on the corrected image data according to the rectangular area of ​​the coordinate position of the masking region, and generates a masking image as a result.

[0051] In S509, the main processing unit 421 requests the print instruction unit 425 to print the masking image generated in S508. The print instruction unit 425 prints the masking image via the print execution unit 413. In this embodiment, the masking application generates a printed copy of the masking image by printing, but the use of the masking image is not limited to this. The masking image may be saved to the HDD 214 via the internal data storage unit 412. The main processing unit 421 may also save the masking image file to the external storage 120 via the internet access unit 426, or send the masking image to any recipient via email. The masking image may be saved using any means.

[0052] 《Explanation of the preview screen》 The preview screen displayed in S507 in Figure 5 will be explained using Figure 8. The preview screen 800 includes a preview image display area 801, a display zoom button 803, a display zoom button 804, a display fit button 805, a print button 806, a basic menu 810, and a confidence level menu 820.

[0053] The preview image display area 801 displays the corrected image data generated in S503 of Figure 5 and the masking area. As shown in the preview masking area 802, the masking area is superimposed on the corrected image data, allowing the user to confirm which areas of the image will be masked before masking is performed. In S507 of Figure 5, when the preview is first displayed, the results of the masking items detected in S505 are displayed, and the masking display is updated according to subsequent user operations.

[0054] The display zoom button 803 increases the display magnification of the preview image display area 801 by a certain amount, making it larger. The display zoom out button 804 decreases the display magnification of the preview image display area 801 by a certain amount, making it smaller. The display fit button 805 changes the magnification to the maximum that fits the corrected image data within the preview image display area 801. In accordance with any of the display magnification operations, the preview masking area 802 is updated to follow the corresponding position and size on the corrected image data.

[0055] The basic menu 810 is a menu that provides operation menus for editing the masking area. The basic menu 810 includes an area add button 811, an area delete button 812, an area pin button 813, an undo button 814, and an advance button 815.

[0056] The Add Region button 811 is used to add a new masking region. The user first presses the Add Region button 811, and then touches the text region to be added to the masking target on the preview image display area 801. Based on the touched coordinates and the magnification of the preview display, the touched coordinates on the corrected image data are obtained. Then, based on the touched coordinates and the BS / OCR results obtained in S504 in Figure 5, the touched text region is identified. The identified text region is determined as a new masking region and is added and displayed as a new preview masking region in the preview image display area 801. In this way, the Add Region button 811 makes it possible to add a masking region even if no masking item was detected in S505.

[0057] The area deletion button 812 is used to delete the masking area. When the area deletion button 812 is pressed, and a preview masking area is subsequently touched on the preview image display area 801, the corresponding preview masking area is deleted. In this way, the area deletion button 812 makes it possible to delete an area that was mistakenly detected as a masking item in S505, by removing it from the masking process.

[0058] The area pin button 813 is a button used to fix the state in which a preview masking area is being masked. It is a button used to fix the state so that it does not change from being masked to not being masked during operation of the confidence menu 820. When the user presses the area pin button 813 and then touches a preview masking area on the preview image display area 801, the state of the corresponding preview masking area is changed to the pinned state. The pinned state may be indicated on the preview image display area 801. For example, the fill color or border color of the preview masking area may be changed, or an icon indicating the pinned state may be displayed overlaid on the preview masking area or displayed near it.

[0059] The undo button 814 cancels a user action and returns to the previous state according to the user action record on this preview screen. The forward button 815 advances to the next state from the currently displayed preview screen. Depending on the user's actions, these buttons may be controlled to be unpressable. The undo button 814 and forward button 815 make it easy to return to a previous state in case of an error on the preview screen.

[0060] The confidence level menu 820 is a menu that provides a menu for switching masking targets using confidence levels. The confidence level menu 820 includes an attribute type selection box 821 and a slider 822. The attribute type selection box 821 is a checkbox for the user to specify the target of the confidence level-based masking target switching operation. The slider 822 is an object for switching whether or not to mask areas that are not pinned among the masking areas corresponding to the attribute type specified in the attribute type selection box 821, using confidence levels. With the slider 822, it is possible to specify a masking level, which is a measure of whether or not to comprehensively mask personal information or confidential information, within a certain range. The higher the masking level, the more areas with low confidence levels will be masked. The lower the masking level, the less areas with low confidence levels will be masked. When the system receives instructions from the user via the slider 822, the masking target determination unit 431 executes a process to determine the masking targets using confidence levels. The process to determine the masking targets using confidence levels will be described later.

[0061] The print button 806 is used to print after masking the corrected image data. When the display control unit 427 detects that the print button 806 has been pressed, it determines the masking area from the state of the preview masking area displayed in the preview image display area 801, and the process proceeds to S509. In S509, the specified mask is applied to the masking area and printing is performed, and the process shown in the flowchart in Figure 5 is completed.

[0062] Explanation of the process for determining what to mask using confidence levels. The process of determining the masking target using confidence, which is performed by the masking target determination unit 431, will be explained using the flowchart in Figure 9 and Figure 10. The flowchart in Figure 9 is the process that is executed when the user gives instructions via the slider 822 on the preview screen displayed in S507 of Figure 5.

[0063] In S901, the masking target determination unit 431 acquires user input obtained through the preview screen. Specifically, the masking target determination unit 431 acquires the state of the slider 822 in Figure 8, the attribute type selection box 821, and the pinning status of each region instructed by the user via the region pinning button 813.

[0064] In S902, the masking target determination unit 431 acquires the group of items to be processed in this process. If a uniform confidence threshold is used to determine whether or not to mask all items, there is a possibility that the masking process may be switched for items that should not be changed. Therefore, in S902, the group of items to be processed is narrowed down to those items that are not pinned among the masking items that correspond to the attribute type specified in the attribute type selection box 821.

[0065] In S903, the masking target determination unit 431 determines a confidence threshold that serves as the criterion for determining whether an item is to be masked. In this embodiment, the confidence threshold is determined from the slider 822 in Figure 8. Figure 10(a) shows an example of the slider 822, and the masking level can be obtained from the position specified by the slider. Figure 10(b) shows an example of the process of determining a confidence threshold that switches whether an item is to be masked or not based on the masking level. The masking level and the confidence threshold are represented by a graph that decreases linearly, as shown by line 1002 in Figure 10(b). As a result, when the masking level is high, even if the confidence level of an item is low, it is determined to be to be masked in order to comprehensively mask personal or confidential information. For example, if a masking level of 1000 is specified, the confidence threshold is determined to be 0.3. If a masking level of 1001 is specified, the confidence threshold is determined to be 0.7. Furthermore, the method for determining the confidence threshold from the masking level is not limited to the graph shown in Figure 10(b), but may also involve a nonlinear relationship such as a step function or inverse proportion.

[0066] In S904, the masking target determination unit 431 selects one item from the group of items to be processed acquired in S902 that has not yet undergone determination processing. In S905, the masking target determination unit 431 performs a threshold determination using the confidence level of the item to be processed selected in S904 and the confidence level threshold determined in S903. If the confidence level is equal to or greater than the threshold, in S906 the masking target determination unit 431 determines that the item is to be masked (masking string). If the confidence level is less than the threshold, in S907 the masking target determination unit 431 determines that the item is not to be masked. Figures 10(c) and 10(d) show the preview display area of ​​the preview screen 800 when masking level 1000 and masking level 1001 are specified, respectively. Figure 10(c) shows an example where items 1003 and 1004, detected as postal codes, and item 1005, detected as an address, all have a confidence level of 0.3 or higher, determined from the confidence threshold of masking level 1000, and are therefore determined to be masked. Figure 10(d) shows an example where item 1006, which was incorrectly detected as a postal code, does not meet the condition of a confidence level of 0.7 or higher, determined from masking level 1001, and is therefore not subject to masking, resulting in a change in display. The decision of whether or not to mask an item is made by a binary decision based on a thresholding process for the confidence level, but the intensity of the color display for items where the difference between the confidence level and the threshold decreases may be gradually changed according to the specified masking level. Gradually changing the intensity of the color display allows the user to operate the slider while keeping track of which item is likely to be masked next, thus improving the usability when switching masked items. Furthermore, visibility may be further improved by surrounding items above the threshold with a solid line frame and items below the threshold with a dotted line frame.

[0067] In S908, the masking target determination unit 431 determines whether threshold determination has been completed for all of the items to be processed acquired in S902. If determination has not been completed for all of the items to be processed, the process returns to S904. If determination has been completed for all of the items to be processed, the process shown in the flowchart in Figure 9 is completed.

[0068] In this way, by accepting the user's specification of the masking level using the slider 822 (object) through user operation, the masking process can be uniformly decided while confirming the decision result of whether or not to mask an item according to its confidence level. This eliminates the need to specify whether or not to mask each individual item. In this embodiment, an example of performing the operation of deciding which items to mask using a slider for specifying the masking level has been described, but this method is not limited to this. For example, the confidence level threshold may be specified directly by user operation without using the concept of a masking level. Alternatively, a checkbox may be provided to accept instructions to uniformly exclude items with low confidence levels from masking using a pre-held confidence level threshold. Furthermore, in this embodiment, an example of handling continuously changing confidence levels has been shown, but depending on the method of obtaining the confidence level, a stepped confidence level may be obtained, such as two levels, "high" and "low," or three levels, "high," "medium," and "low." In that case, by accepting the masking level in a stepped manner, instructions may be accepted on which level of confidence level items should be masked.

[0069] As described above, the technology of this embodiment makes it possible to reduce the burden on the user in the masking area verification process. Specifically, by providing a display control unit that displays whether or not to mask an item based on the confidence level of the masking item, the user does not need to individually specify which items to mask, and can efficiently obtain the masked output.

[0070] [Second Embodiment] In the first embodiment, an example was described in which a threshold is applied to the confidence value when deciding whether or not to mask an item based on the confidence level of the automatically recognized masking item. However, if the confidence level is a real number and there are items with similar values, fine adjustment with a slider may be necessary, which may reduce usability. Therefore, in this embodiment, instead of using the confidence level of the items as is, an example is described in which the likelihood of items to be masked is ranked based on the confidence level, and the user is asked to specify whether or not to mask an item based on that ranking. When describing this embodiment, the description of parts that are the same as the first embodiment in terms of configuration and processing procedure will be omitted, and only the parts that differ will be described.

[0071] Figure 11 is a flowchart of this embodiment for generating and printing a masking image from an image scanned by the MFP 110. The differences from the flowchart in the first embodiment shown in Figure 5 will be explained. In S1101, the main processing unit 421 requests the confidence acquisition unit 430 to rank all masking items detected in S505 based on the confidence level acquired in S506. The confidence acquisition unit 430 stores the ranking results based on the confidence level for all masking items in the RAM 213.

[0072] Figure 12 is a flowchart of the ranking process. In S1201, the confidence acquisition unit 430 acquires the items to be ranked. If the process in S1201 is executed in the process of S1101, the items to be ranked are all masked items detected in S505. In S1202, the confidence acquisition unit 430 acquires the confidence level of the items to be ranked. The confidence level acquired in S506 can be used as is. In S1203, the confidence acquisition unit 430 ranks the items to be ranked based on the confidence level. Specifically, by ranking in descending order of confidence level, the items can be ranked in descending order of confidence level. Returning to the explanation of Figure 11.

[0073] In S1102, the main processing unit 421 requests the display control unit 427 to generate a preview screen using the corrected image data generated in S503, and the masking items and their ranking extracted in S505 and S1101. The process for determining the masking target in this embodiment will be explained with reference to Figures 13 and 14.

[0074] Figure 13 is a flowchart of the process for determining the masking target using confidence level in this embodiment. The process in the flowchart of Figure 13 is executed by the masking target determination unit 431. Below, only the differences from Figure 9 will be explained. In S1301, the masking target determination unit 431 determines the rank threshold used to determine the masking target based on the user instruction obtained in S901 and the group of items to be processed and the number of items obtained in S902.

[0075] In S1302, the masking target determination unit 431 ranks the items to be processed, acquired in S902, based on their confidence level. Specifically, the items should be ranked in descending order of their confidence level, with the items being ranked in descending order of their confidence level. In S1303, the masking target determination unit 431 performs a threshold determination using the rank of the items to be processed selected in S904 and the rank threshold determined in S1301. If the rank is equal to or greater than the threshold, the masking target determination unit 431 determines that the item will be masked in S906. If the rank is less than the threshold, the masking target determination unit 431 determines that the item will not be masked in S907.

[0076] Figure 14(a) shows an example of the slider 822 in this embodiment. Here, the slider is used to specify the masking level in steps, which is a measure of whether or not to comprehensively mask personal or confidential information, as in the first embodiment. The number of steps on the slider is determined according to N, the number of ranked masking items, and the number of steps is N+1. When the masking level is set to the highest step on the slider, all items are subject to masking. When the masking level is set to the second highest step on the slider, items up to N-1 in descending order of confidence level are subject to masking. This achieves the behavior that items with lower confidence levels are excluded from masking each time the slider is moved down one step.

[0077] Figures 14(b) and 14(c) show the preview display area of ​​the preview screen 800 when masking levels 1401 and 1402 are specified, respectively. Figure 14(b) shows an example where all items are subject to masking. Figure 14(c) shows an example where the two items with the lowest confidence level are not subject to masking. "123-0045", which was incorrectly detected as a postal code, is displayed as not subject to masking from item 1403 to item 1406, and "Ota-ku, Tokyo", which was detected as an address, is displayed as not subject to masking from item 1405 to item 1407. In this way, the slider can be used to determine whether or not to subject masking items according to the ranking of confidence levels in stages. As in the first embodiment, it is possible to limit the items to be switched using the pin button 813 and the attribute type selection box 821. In that case, the items to be ranked are narrowed down and the number of stages displayed as the slider is updated.

[0078] In this embodiment, we have described an example in which items to be masked are ranked based on their confidence level, and the user is allowed to specify whether or not to mask an item based on that ranking. This makes it possible to provide the user with the option to switch the items to be masked using their confidence level, even when there are items with similar real-value confidence levels, thereby reducing the burden on the user during the masking area verification process.

[0079] [Third Embodiment] In the third embodiment, we will describe the switching of the masking target when the extracted string has multiple different attribute types. In such an example, the masking area (masked string) may change when the "Attribute Type to Mask" item selected in the confidence menu in Figure 8 is changed. In this embodiment, we assume that the extracted string "123-0045" has two attribute types: "Postal Code" and "Document Number". Also, for the sake of brevity, we will set the confidence threshold to 0.7.

[0080] The outline of this embodiment will be described with reference to Figures 15 and 16. As shown in Figures 15(a) and 16(a), the attribute types to be masked include "document number" in addition to the attribute types to be masked in the first embodiment. Also, as shown in Figures 15(a) and 16(a), multiple different attribute types are selected by the selection unit 432.

[0081] The example in Figure 15, as shown in Figure 15(a), is an example where items other than "Document Number" are checked in the attribute type specification checkbox 1503. In the example in Figure 15(a), "Document Number" is not an attribute type to be masked, so the confidence calculation result is as shown in Table 5.

[0082] [Table 5]

[0083] As shown in Table 5, the string "123-0045" has two attribute types: "postal code" and "document number". In the example in Figure 15, the attribute type "document number" is not masked, so the confidence score for the attribute type "document number" in the string "123-0045" is not calculated. Therefore, the string "123-0045" has a confidence score of 0.63 for the attribute type "postal code".

[0084] Figure 15(b) shows the result of the preview image. Since the confidence threshold is 0.7, the confidence level of the string "123-0045" is below the threshold, and therefore the string "123-0045" is not to be masked and will be filled with a lighter density than the masking area.

[0085] Next, we will explain the example in Figure 16(a). In the example in Figure 16(a), all items are checked in the attribute type specification checkbox 1603, as shown in Figure 16(a). In the example in Figure 16(a), "Document Number" is the attribute type to be masked, so the confidence level calculation result is as shown in Table 6.

[0086] [Table 6]

[0087] As shown in Table 6, the string "123-0045" has two attribute types: "postal code" and "document number". In the example in Figure 16(a), the confidence level of the string "123-0045" as an attribute type "postal code" is 0.63, and as an attribute type "document number" is 0.81. In such cases, the string is treated as the attribute type with the highest confidence level. In the example in Figure 16(a), the attribute type of the string "123-0045" is "document number", and the confidence level of the string "123-0045" is 0.81. In other words, when multiple different attribute types are selected by the selection unit 432, the confidence level of the string is obtained for each selected attribute type. The string is treated as the attribute type with the highest confidence level, and the confidence level of the string becomes the confidence level with the highest value.

[0088] Figure 16(b) shows the result of the preview image. Since the confidence threshold is 0.7, the confidence level of the string "123-0045" is above the threshold, and the string "123-0045" is targeted for masking and is masked in the same way as the other masking regions. Thus, when the selection item for "Attribute type to be masked" is different, the same string may or may not be targeted for masking.

[0089] As described above, if the extracted string has multiple different attribute types, the selection in the "Attribute Type to Mask" item will determine whether or not the string is masked. Even when the extracted string has two or more different attribute types, the UI screens shown in Figures 15 and 16 make it easy to switch the masking target. This reduces the burden on the user when checking the masked area.

[0090] Other embodiments This disclosure can also be realized by performing the following process: supplying software (programs) that realize the functions of the embodiments described above to a system or device via a network or various storage media, and having the computer (or CPU, MPU, etc.) of that system or device read and execute the program. Furthermore, the above process can also be realized by a circuit (e.g., an ASIC) that realizes one or more functions.

[0091] Furthermore, while the above-described embodiment cited a masking application installed and executed on an MFP as an example of an application, the method is not limited to this example and can be implemented and is effective with any application that has similar masking functionality.

[0092] Furthermore, in the embodiments described above, a personal computer was assumed as the information processing device. However, the information processing device is not limited to the examples described above. The information processing device may be, for example, a mobile phone, a personal digital assistant, a digital still camera, a digital video camera, a portable music player, a game console, a set-top box, an internet-connected home appliance, etc. Moreover, this disclosure can be implemented for any information processing device (terminal) that can be used in a similar manner.

[0093] Furthermore, while the embodiments described above used wired LAN or wireless LAN as examples of network configurations, the invention is not limited to these examples. For example, any other network configuration such as IEEE1394 or Bluetooth® may be used.

[0094] Although preferred embodiments of this disclosure have been described in detail above, the present invention is not limited to these specific embodiments, and various modifications or changes are possible within the scope of the gist of the invention as described in the claims.

[0095] The above-described embodiments include the following configurations.

[0096] (Configuration 1) An information processing device for performing string masking processing, comprising: extraction means for extracting strings from an input document image; selection means for selecting, based on user instructions, the attribute types to be targeted for the masking processing from among the attribute types of strings contained in the document image; acquisition means for obtaining the confidence level that each string contained in the document image is of the selected attribute type; processing means for performing the masking processing on the region of the document image in which the confidence level is above a threshold; and display control means for displaying a preview image of the masked document image on a display means.

[0097] (Configuration 2) The information processing device according to Configuration 1, characterized in that the threshold is determined based on user input.

[0098] (Configuration 3) The information processing apparatus according to Configuration 1 or 2, wherein, if there are multiple different selected attribute types in each string, the acquisition means acquires the confidence level of each string for each selected attribute type, and sets the confidence level showing the highest value as the confidence level of each string.

[0099] (Configuration 4) The information processing apparatus according to any one of Configurations 1 to 3, wherein the display control means displays an object that continuously changes the threshold on the user interface screen that displays the preview image.

[0100] (Configuration 5) The information processing apparatus according to any one of Configurations 1 to 3, wherein the display control means displays an object that changes the threshold in steps on the user interface screen that displays the preview image.

[0101] (Configuration 6) The information processing apparatus according to any one of Configurations 1 to 5, characterized in that the display control means changes the density of the color used to mask the string according to the difference between the confidence level of the string and the threshold.

[0102] (Configuration 7) The information processing apparatus according to Configuration 6, wherein the display control means makes the masking color darker as the confidence value of the string increases.

[0103] (Configuration 8) The information processing apparatus according to any one of Configurations 1 to 7, characterized in that the display control means sets the color used for the masking process to black.

[0104] (Configuration 9) The information processing apparatus according to any one of Configurations 1 to 7, characterized in that the display control means uses the color used for the masking process as the background color.

[0105] (Configuration 10) The information processing apparatus according to any one of Configurations 1 to 9, characterized in that the display control means surrounds the string with a solid line when the confidence level of the string is equal to or greater than the threshold.

[0106] (Configuration 11) The information processing apparatus according to any one of Configurations 1 to 9, characterized in that the display control means surrounds the string with a dotted line when the confidence level of the string is less than the threshold.

[0107] (Configuration 12) An information processing device according to any one of Configurations 1 to 11, characterized in that it prints the preview image displayed on the display means.

[0108] (Configuration 13) An information processing device according to any one of Configurations 1 to 11, characterized in that it outputs the preview image displayed on the display means to a file.

[0109] (Configuration 14) A control method for an information processing device that performs string masking, comprising: a step of extracting strings from an input document image; a step of selecting, based on user instructions, an attribute type to be the target of the masking process from among the attribute types of strings contained in the document image; a step of obtaining a confidence level for each string contained in the document image that it is of the selected attribute type; a step of performing the masking process on the region of the document image in which the confidence level is equal to or greater than a threshold; and a step of displaying a preview image of the masked document image on a display means.

[0110] (Configuration 15) A program for causing a computer to function as an information processing device as described in any one of Configurations 1 to 13.

Claims

1. An information processing device that performs string masking, An extraction means for extracting text from an input document image, A selection means for selecting, based on user instructions, the attribute types of the strings contained in the document image to be subject to the masking process, A means for obtaining the confidence level that each string contained in the document image is of a selected attribute type, Processing means for performing the masking process on the region of the document image in which the confidence level is equal to or greater than a threshold, A display control means for displaying a preview image of the masked document image on a display means, An information processing device characterized by having the following features.

2. The information processing apparatus according to claim 1, characterized in that the threshold is determined based on user input.

3. The information processing apparatus according to claim 1, wherein, if there are multiple different selected attribute types in each string, the acquisition means acquires the confidence level of each string for each selected attribute type, and sets the confidence level showing the highest value as the confidence level of each string.

4. The information processing apparatus according to claim 1, wherein the display control means displays an object that continuously changes the threshold on the user interface screen that displays the preview image.

5. The information processing apparatus according to claim 1, wherein the display control means displays an object that changes the threshold in steps on the user interface screen that displays the preview image.

6. The information processing apparatus according to claim 1, characterized in that the display control means changes the density of the color used to mask the string according to the difference between the confidence level of the string and the threshold.

7. The information processing apparatus according to claim 6, characterized in that the display control means makes the masking color darker as the confidence value of the string increases.

8. The information processing apparatus according to claim 1, characterized in that the display control means sets the color used for the masking process to black.

9. The information processing apparatus according to claim 1, characterized in that the display control means uses the color used for the masking process as the background color.

10. The information processing apparatus according to claim 1, characterized in that the display control means surrounds the string with a solid line when the confidence level of the string is equal to or greater than the threshold.

11. The information processing apparatus according to claim 1, characterized in that the display control means surrounds the string with a dotted line when the confidence level of the string is less than the threshold.

12. The information processing apparatus according to claim 1, characterized in that it prints the preview image displayed on the display means.

13. The information processing apparatus according to claim 1, characterized in that it outputs the preview image displayed on the display means to a file.

14. A control method for an information processing device that performs string masking, The process involves extracting text from an input document image, A step of selecting, based on user instructions, the attribute types to be subjected to the masking process from among the attribute types of strings contained in the document image, The process of obtaining the confidence level that each string contained in the aforementioned document image is of the selected attribute type, The process of performing the masking on the region of the document image in which the confidence level is equal to or greater than a threshold, The steps include: displaying a preview image of the masked document image on a display means; A control method for an information processing device, characterized by having the following features.

15. A program for causing a computer to function as an information processing device according to any one of claims 1 to 13.

Images