Information processing system and its control method, program

Abstract

It is possible to easily determine whether a document containing a background pattern is a forgery or not. [Solution] A first image is obtained in which no specific image processing to make a background pattern appear has been performed on an image containing a predetermined document, and a second image is obtained in which the specific image processing to make a background pattern appear has been performed on the image containing the predetermined document. Based on the obtained first and second images, it is determined whether or not the predetermined document is the original.

Description

Technical Field

[0001] The present invention relates to an information processing system, a control method thereof, and a program.

Background Art

[0002] With the progress of digitization in the world, there is a mechanism for electronically exchanging documents (e.g., household register extracts) including watermarks for purposes such as identity verification.

[0003] In Patent Document 1, a technique is disclosed in which watermark information obtained from detected document information or image information is compared with electronic watermark information recorded in the first information management database to determine whether it matches the electronic watermark information registered in the first information management database, and based on the output result, the authenticity of a special document is determined.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Disclosure of the Invention

Problems to be Solved by the Invention

[0005] When electronically exchanging a document including a watermark, one concern is that the document may be forged. Therefore, the recipient of a document including a watermark needs to confirm that the document has not been forged (including not being copied, i.e., being the original).

[0006] Patent Document 1 discloses a mechanism for determining authenticity by comparing image information with electronic watermark information recorded in a database, but does not disclose a mechanism for determining authenticity without using the electronic watermark information recorded in the database. In other words, a mechanism for determining authenticity only from a photographed image is not disclosed.

[0007] Therefore, the present invention aims to provide a mechanism for easily determining whether a document containing a background pattern is a forgery or not. [Means for solving the problem]

[0008] To solve the above problems, the present invention provides A first acquisition means for acquiring a first image in which no specific image processing has been performed to make a background pattern appear on an image containing a predetermined document, A second acquisition means for acquiring a second image in which specific image processing is performed on the image containing the predetermined document to make a background pattern stand out, A determination means for determining whether the predetermined document is the original based on the first image and the second image acquired by the first acquisition means and the second acquisition means, It is characterized by being equipped with [the following features]. [Effects of the Invention]

[0009] According to the present invention, it is possible to easily determine whether a document containing a background pattern is a forgery or not. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows an example of a system configuration. [Figure 2] This figure shows an example of the hardware configuration of various devices. [Figure 3] This flowchart shows an example of the processing of the first embodiment of the counterfeit detection system. [Figure 4] This figure shows an example of a counterfeit detection system. [Figure 5] This diagram shows examples of cases where an item is determined to be an original and cases where it is determined to be a forgery. [Figure 6] This flowchart shows an example of the processing of the second embodiment of the counterfeit detection system. [Figure 7]It is a diagram showing an example of a case determined as an original and a case determined as a forgery in the second embodiment. [Figure 8] It is a flowchart showing an example of a case where the first embodiment and the second embodiment are treated as a series of processes.

Mode for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0012] In the present embodiment, an example of determining whether or not documents related to a household register (for example, a household register transcript) have been forged in an inheritance procedure will be used for explanation.

[0013] When an heir inherits assets deposited by the decedent with a securities company, it is necessary to go through an inheritance procedure. In that inheritance procedure, the heir needs to send documents related to the household register to the securities company to prove the relationship between the heir and the decedent.

[0014] Conventionally, it has been done by mail, but it has been time-consuming to prepare for that mail. In addition, when the decedent has deposited assets with a plurality of securities companies, the procedural burden on the heir has been even higher.

[0015] Therefore, by sending an image of the original of the document related to the household register, it becomes possible to reduce that burden. However, the possibility of a forged document being sent also increases.

[0016] Therefore, using the system in the present embodiment, it is determined whether or not a document related to the household register has been forged (whether the document is an original).

[0017] Referring to FIG. 1, an example of the configuration of the information processing system in the present embodiment will be described.

[0018] In FIG. 1, the client terminal 101 and the server 102 are communicably connected via a network. Note that these are just examples, and other configurations such as a configuration that is completed only with the client terminal 101 may be possible.

[0019] The client terminal 101 is an information processing device such as a personal computer (hereinafter referred to as a PC), a smartphone, a tablet, etc. The client terminal 101 may be a device equipped with an imaging mechanism or a device that can be connected to an imaging device even if it does not have an imaging mechanism.

[0020] The server 102 is an information processing device capable of executing forgery determination processing, etc. on the acquired image. One server may be used, or it may be composed of a plurality of servers. Also, it may be in a cloud environment or an on-premises environment.

[0021] In this embodiment, an example in which the server 102 executes the processing as the main body will be described, but the client terminal 101 may execute the processing as the main body. Also, in this embodiment, an example in which processing including shooting, etc. can be executed on a web browser without installing a predetermined native app on the client terminal 101 will be described, but a predetermined native app for executing the system in this embodiment may be installed on the client terminal 101.

[0022] Next, referring to FIG. 2, an example of the configuration of the client terminal 101 and the server 102 as an example of a device applicable in this embodiment is shown. Note that the server 102 may not have some configurations such as the display 205 and the camera unit 212.

[0023] In FIG. 2, the CPU 201, the memory 202, the non-volatile memory 203, the image processing unit 204, the display 205, the operation unit 206, the recording medium I / F 207, the external I / F 209, and the communication I / F 210 are connected to the internal bus 250. Each unit connected to the internal bus 250 is configured to be able to exchange data with each other via the internal bus 250.

[0024] Memory 202 consists of, for example, RAM (volatile memory using semiconductor elements).

[0025] The CPU 201 controls the operator terminal 101, server 103, and supervisor terminal 104, for example, by using memory 202 as work memory, according to a program stored in non-volatile memory 203. Non-volatile memory 203 stores image data, audio data, other data, and various programs necessary for the CPU 201 to operate. Non-volatile memory 203 is composed of, for example, a hard disk (HD) or ROM. In addition to the CPU, other processing units such as GPUs and TPUs may also be included.

[0026] The image processing unit 204 performs various image processing operations on image data stored in the non-volatile memory 203 and recording medium 208, video signals acquired via the external I / F 209, image data acquired via the communication I / F 210, and captured images, based on the control of the CPU 201. The image processing operations performed by the image processing unit 204 include A / D conversion, D / A conversion, image data encoding, compression, decoding, resizing, noise reduction, and color conversion. The image processing unit 204 may be composed of dedicated circuit blocks for performing specific image processing operations. Furthermore, depending on the type of image processing, the CPU 201 may perform image processing according to a program without using the image processing unit 204.

[0027] The display 205 displays images and GUI (Graphical User Interface) screens based on the control of the CPU 201. The CPU 201 generates display control signals according to the program and controls the various parts of the operator terminal 101 and supervisor terminal 104 to generate video signals for display on the display 205 and output them to the display 205. The display 205 displays images based on the output video signals. Note that the operator terminal 101 and supervisor terminal 104 themselves are configured to have an interface for outputting video signals for display on the display 205, and the display 205 may be configured as an external monitor (such as a television).

[0028] The operation unit 206 is an input device for receiving user input, including text input devices such as keyboards, pointing devices such as mice and touch panels, buttons, dials, joysticks, touch sensors, and touchpads. The touch panel is configured planarly by being superimposed on the display 205, and is an input device that outputs coordinate information corresponding to the position of contact.

[0029] The recording medium interface (I / F207) allows for the insertion of recording media 208 such as memory cards, CDs, and DVDs. Based on the control of the CPU 201, it reads data from the inserted recording media 208 and writes data to the recording media 208. The external interface (I / F209) is an interface for connecting to external devices via wired or wireless cables and for inputting and outputting video and audio signals. The communication interface (I / F210) is an interface for communicating with external devices and the internet 211 to send and receive various data such as files and commands.

[0030] The camera unit 212 is a camera unit composed of an image sensor (imaging sensor) such as a CCD or CMOS element that converts an optical image into an electrical signal.

[0031] Next, with reference to Figure 3, an example of the processing of the counterfeit detection system in the first embodiment is shown. The processing of each step is executed by the CPU 201 of the server 102. When a request to activate the camera is received from the client terminal 101, the processing of this flowchart is started.

[0032] In S301, CPU201 activates the camera on the web browser of client terminal 101. Specifically, when it receives an operation indicating the start of the procedure (or start of shooting) on ​​the website for inheritance procedures, it starts processing this flowchart. Subsequently, it controls the web browser of client terminal 101 to activate the camera. Also, of the images taken by client terminal 101, only one can be sent to server 102. The reason for limiting the number of images taken to one (one shooting count) will be explained later.

[0033] In other words, it is an example of a means for receiving an image containing a predetermined document. In other words, the image containing the predetermined document is an example of an image captured by an imaging means.

[0034] Furthermore, the client terminal 101 is controlled so that it cannot select an image different from the one captured in this step. For example, it may be possible to display an image pre-stored in the client terminal 101 but prevent it from being selected, or it may be possible to display only the image captured in this step and allow the user to choose whether or not to send that image. In other words, this is an example of a means of receiving an image captured by the imaging means, without receiving an image stored in an electronic device. As a result, if an image different from the one captured on the spot is used, the risk of it being forged increases, so this risk can be reduced by having the image captured on the spot.

[0035] In the first embodiment, an example of activating the camera on a web browser is described, but this method is not the only one that can be used. Other methods are also acceptable as long as the camera on the client terminal 101 can be activated, such as controlling the client terminal 101 to launch a camera application installed on it.

[0036] In the first embodiment, an example was described in which control is made so that an image different from the one captured in this step cannot be selected. However, this is not the only method, and other methods are also acceptable as long as the selected image can be transmitted, such as accepting selection from images pre-stored in the client terminal 101 or accepting selection from captured images.

[0037] Specifically, the system allows the user to specify whether to start shooting or upload a document. If the user requests to start shooting, the system executes the process described in this step. If the user requests to upload a document, the system accepts the user's selection of the image to be uploaded from the images stored in the non-volatile memory 203 of the client terminal 101.

[0038] In S302, CPU201 receives an image of a document containing a watermark. That is, it receives an image of a document containing a watermark that was captured by client terminal 101.

[0039] A background pattern refers to a design or a set of characters applied to the background of paper. For example, a predetermined set of characters embedded in the background of paper for anti-counterfeiting purposes is called a background pattern. These predetermined characters could be, for instance, characters indicating that the document is a copy, such as "Reproduction (Prohibited)" or "Copy (Prohibited)." In other words, the predetermined characters are difficult to see on the original document, but when the original is copied, the predetermined characters become visible on the copy. This makes it easy to determine whether the document is the original or not.

[0040] Furthermore, the background pattern may also be referred to by other names such as hidden text, watermark text, or microtext, and can be any pattern or arbitrary string of characters applied to the background of the paper. In addition, in the first embodiment, it can be any string of characters printed on the paper that becomes visible through scanning or image processing.

[0041] Documents containing a watermark are, for example, documents issued by public institutions. Examples include family register transcripts, family register extracts, resident registration certificates, seal registration certificates, tax certificates, and tax payment certificates. However, these are merely examples and do not necessarily imply that the document must be issued by a public institution. Any other document containing a watermark is acceptable.

[0042] Furthermore, if a document is received that does not contain a watermark or is not a specified document, or if the system in the first embodiment cannot determine whether it is an original document, an error notification may be output. For example, the types of documents that can be received can be predetermined, and an error can be output if any other type of document is received.

[0043] In S303, CPU201 acquires a first image in which no specific image processing has been applied to the received image. In other words, this step is an example of a process that acquires a first image in which no specific image processing has been performed to make a background pattern appear on an image containing a predetermined document. The specific image processing will be explained in S304.

[0044] "Not applying specific image processing" means skipping the specific image processing required to highlight the background pattern contained in the document. In other words, image processing such as blur correction and distortion correction is performed on the received image for subsequent processing, but the specific image processing required to highlight the background pattern is skipped. The first image, which has not undergone specific image processing in this manner, is stored in a temporary area or memory 202 for result determination.

[0045] While we have described skipping a specific image processing step as an example of not applying a specific image processing step, this method is not the only option. Other methods are also acceptable, such as performing image processing other than the specific image processing step, as long as the received image is not subjected to the specific image processing step.

[0046] For example, the received image is stored as the first image (image processing such as blur correction may be performed as it will be used in subsequent processing) in memory 202 for result determination. In other words, instead of performing a skip operation, the received image is stored as is in the temporary area or memory 202. After that, the received image is duplicated, and the processing in the subsequent steps is performed on the duplicated image.

[0047] In S304, CPU201 performs specific image processing on the received image. This specific image processing is designed to make the background pattern contained in the document stand out. In other words, just as the background pattern becomes visible when a document containing a background pattern is scanned using a copier, CPU201 performs image processing to achieve a similar result.

[0048] For example, a background pattern can be made visible by performing a binarization process. The background pattern is made up of faint pixels that are difficult to see with the naked eye. The background pattern is made visible by performing a binarization process based on the value of each pixel in the image and a set threshold, so that the pixels that make up the background pattern become visible. The threshold can be determined as a fixed value, or it can be adjusted using known techniques such as mode analysis or discriminant analysis.

[0049] Furthermore, the process is not limited to binarization; any other image processing method that brings out the background pattern contained in the document, such as adjusting the edge gradient, hue, saturation, brightness, or lightness of the image, is also acceptable.

[0050] In other words, this is an example of how specific image processing that makes it difficult to recognize background patterns can be any process that adjusts at least one of the following: edge gradient, hue, saturation, brightness, or luminance of an image, or at least one of the processes that removes shadows from an image.

[0051] In S305, CPU201 acquires a second image that has undergone specific image processing in S304. Specifically, it stores the second image in the temporary area for result determination where the first image is stored, or in memory 202.

[0052] In other words, this step is an example of a process that obtains a second image in which specific image processing has been performed on an image containing a given document to make a background pattern stand out.

[0053] In S306, CPU201 determines whether the image received in S302 is a forgery or not. If it is not a forgery (it is the original), proceed to S307; otherwise, proceed to S308.

[0054] In other words, this step is an example of a process that determines whether a given document is the original based on the acquired first and second images.

[0055] Specifically, the system determines whether or not an image is a forgery based on the first and second images. That is, it determines whether or not an image is a forgery based on the first and second images stored in a temporary area or memory 202 for result determination. The forgery determination will be explained using Figures 4 and 5. Figure 4 is a conceptual diagram showing the process in the first embodiment.

[0056] The system receives an image of the family register transcript 401 taken by the client terminal 101 (S302), and obtains a first image 402 (S303) of the family register transcript 401 without any specific image processing, and a second image 403 (S305) with specific image processing applied. The word "copy" appears in the second image 403 due to the specific image processing. Based on the first image 402 and the second image 403, it is determined whether or not the document is a forgery.

[0057] Figure 5 shows examples of correct patterns and patterns that are judged as forgeries. Figure 5(a) is a pattern that is judged as correct. In the first image 402, the word "copy" is not visible, while in the second image 403, the word "copy" is visible.

[0058] If a family register is the original, scanning it with a copier will reveal the word "copy." Similarly, if a family register is the original, the word "copy," which was difficult to see before performing a specific image processing, will become visible after the processing is completed.

[0059] Therefore, as shown in Figure 5(a), if the first image 402 and the second image 403 are images of the same family register, the word "copy" will not be present in the first image 402, while the word "copy" will be visible in the second image 403, thus confirming that the family register is the original.

[0060] In other words, if no background pattern is visible in the first image, but a background pattern is visible in the second image, this is an example of a means for determining that the specified document is the original.

[0061] Figure 5(b) shows that the word "copy" does not appear in either the first image 402 or the second image 403. This indicates that the received image does not have a watermark embedded in the background. Documents containing watermarks have a watermark determined by each local government embedded in the background of the paper due to the confidentiality of their contents, and are different from ordinary paper sold to the general public. Therefore, if no text appears even after performing specific image processing, it can be confirmed that the received image of the family register is not the original.

[0062] In other words, if the first image does not show a background pattern, and the second image does not show a background pattern, this is an example of a means to determine that the specified document is not the original.

[0063] Figure 5(c) shows that the word "COPY" appears in both the first image 402 and the second image 403. This indicates that the received image is a copy of a document that includes a background pattern. In other words, because an image of a document that is a copy of the original was received, the word "COPY" appears in the first image 402, even before any specific image processing is performed.

[0064] In other words, if the first image does not show a background pattern, and the second image does not show a background pattern, this is an example of a means to determine that the specified document is not the original.

[0065] Thus, if the first image 402 and the second image 403 are images of the same family register, it is possible to determine whether or not the family register has been forged by checking the images before and after the execution of a specific image processing method.

[0066] Furthermore, the client terminal 101 takes only one photograph of the document so that the first image 402 and the second image 403 are images of the same family register. Alternatively, one could take two photographs. For example, the first photograph could be left unprocessed, while the second photograph could be processed to reveal the background pattern. A method could be envisioned to determine whether the received image is forged based on the first and second photographs.

[0067] However, if two images are taken, it's possible that the document taken in the first and second images may not be the same document, and that an image of a non-original document may be sent without being noticed. For example, even if the first image 402 and the second image 403 have a pattern (combination) like that shown in Figure 5(a), it cannot be definitively concluded that the background pattern in the second image 403 was revealed by the specific image processing in the first embodiment. In other words, it's possible that an image of a copy of the original document was sent. If that were the case, while inheritance procedures are supposed to be conducted using the original family register (ensuring authenticity), it would become possible to carry out inheritance procedures even with a copy instead of the original.

[0068] Furthermore, it is possible that forged documents may be sent. For example, an image of a regular piece of paper with the word "copy" written on it may be sent. Even if the first image 402 and the second image 403 follow the pattern shown in Figure 5(a), the first image 402 may be an image of a regular piece of paper (not a document with a background pattern), while the second image 403 may be a processed document with text such as "reproduction" written in the background. Therefore, it becomes impossible to guarantee that the image of the document taken the second time is the original document with a background pattern.

[0069] Thus, if the images are taken in separate sessions, the reliability of the judgment results is compromised from the standpoint of ensuring authenticity and the possibility of forgery. Therefore, by performing the processing in the first embodiment on an image acquired in a single session (a single image), it becomes possible to accurately determine whether or not the document contained in that image is forged.

[0070] In the first embodiment, an example was described in which the first image 402 and the second image 403 are acquired in the pattern shown in Figure 5(c). However, the method is not limited to this method, and it is not necessary to acquire the second image 403 if it is possible to make the user aware that the received image is a forgery (not the original).

[0071] For example, when the first image 402, in which a background pattern is visible, is acquired, control may be set so that no specific image processing is performed on the received image.

[0072] Specifically, if the background pattern is visible in the first image 402, it will be determined to be a forgery. Therefore, a processing step may be added after S303 to determine whether or not the background pattern is visible, so that if the background pattern is visible, the process proceeds to S308, and otherwise, it proceeds to S304.

[0073] For example, if the first image 402, in which a background pattern is visible, is obtained, an error notification may be output. For example, an error message such as "Please send the original copy of the family register" may be output.

[0074] Next, we will explain a method for determining whether a document contained in a received image is forged, based on the first image 402 and the second image 403. The determination is made using a trained model that has been trained on each pattern in Figure 5 and the correct / incorrect labels for each pattern. For example, the model is trained to determine that the pattern in Figure 5(a) is correct, and the other patterns are incorrect. The set of the first image 402 and the second image 403 is input to the trained model thus trained, and the determination result is output.

[0075] In other words, this is an example of a means for determining whether a given document is the original or not, using a trained model that has been trained on combinations of the first and second images and the correct labels for those combinations.

[0076] Alternatively, the difference between the first image 402 and the second image 403 may be compared and the result output. That is, the difference between the first image 402 and the second image 403 will be the presence or absence of a background pattern, so if there is a difference, it may be determined that it is not forged, and if there is no difference, it may be determined that it is forged. In this case, since there is a possibility that a difference will be determined due to parts other than the background pattern, after obtaining the first image 402 and the second image 403, corrections may be applied to make the conditions of each image, such as brightness and luminance, match.

[0077] Alternatively, instead of performing the judgment process, the first image 402 and the second image 403 may be displayed side by side, allowing the user to visually confirm them. In other words, this is one example of a means of controlling the display so that the first image and the second image can be compared.

[0078] Alternatively, the system may mechanically determine whether a document is forged using AI or differential detection, and then display the result for the user to confirm. For example, the display screen (not shown) may show the first image 402 and the second image 403 side by side, and may include the result of the determination of whether it is an original or not, and a confirmation button. When the user performs an action indicating that there is no problem with the content, such as pressing the "confirm button," processing according to the determination result is executed. That is, if the result is that it is an original, the system may proceed to S307 to proceed with the inheritance procedure, or if the result is that it is not an original (forged), the system may proceed to S308 to request mailing. In other words, this is one example of a means of controlling the display of the determination result. In this way, by having a human eye verify the document, it becomes possible to more reliably determine whether documents that need to be prevented from being misused, such as family registers, are forged or not.

[0079] Note that this is just one example of a determination method; any other method is acceptable as long as it can be determined that the background pattern is not visible in one image but is visible in the other image.

[0080] In S307, CPU201 proceeds with the inheritance procedure. That is, since the original document has been confirmed, the securities company proceeds with the necessary processing to move forward with the inheritance. For example, this may involve creating a family tree based on the family register transcript.

[0081] In S308, CPU201 sends a mailing request to the applicant. This is an example of a method to notify the applicant to mail the required documents if it is determined that the submitted documents are not originals.

[0082] Specifically, the system will control client terminal 101 to display a message prompting the user to mail documents, such as, "We were unable to confirm that the family register transcript is the original document, so please mail the necessary documents."

[0083] In the first embodiment, a method for notifying a request for mailing was described, but other methods are also acceptable as long as they can notify the recipient that some action is required because the document contained in the received image is not the original, such as by outputting a notification prompting resubmission or notifying a request for mailing after a certain number of application processing sessions have been performed.

[0084] In S309, CPU201 is added to the list as a person of interest. The applicant is added to the list as a person of interest because they may be a malicious individual who attempted to forge documents.

[0085] Furthermore, since it is possible that an image other than the original was sent by mistake, it may be possible to determine whether or not to record it on the list based on a predetermined threshold and the number of transmissions, or to record it on the list only when certain conditions are met, such as when a clearly forged document has been sent.

[0086] For example, a threshold for the number of transmissions can be set, and if the same person sends potentially forged documents more times than the threshold, they can be recorded as a person of interest. Whether or not it is the same person can be identified from the member's registered ID or applicant name, etc. Alternatively, the client terminal 101 can be recorded as a person of interest (or terminal of interest) based on information that identifies the terminal, such as the serial number.

[0087] For example, a clearly forged document might be one that has been processed to include text equivalent to a watermark, such as "copy," on ordinary paper. If such a document is sent, the applicant will be added to a list of persons of interest.

[0088] Alternatively, you may skip this step and output the mailing request, then terminate the process in this flowchart.

[0089] As described above, according to the first embodiment, it is possible to easily determine whether a document containing a background pattern is a forgery or not.

[0090] Next, a second embodiment of the counterfeit detection system will be described using Figure 6. The first embodiment described an example in which counterfeit detection is performed according to a first image that has not undergone any specific image processing and a second image that has undergone specific image processing. The second embodiment describes an example in which counterfeit detection is performed according to a third image that has undergone image processing to make the background pattern difficult to recognize and a fourth image (which is the same as the second image) that has undergone image processing to make the background pattern easier to recognize.

[0091] Each step is executed by the CPU 201 of server 102. When a request to activate the camera is received from client terminal 101, the processing of this flowchart begins.

[0092] The processing in S601-S602 is the same as that in S301-S302, so it will be omitted.

[0093] In S603, CPU201 performs image processing on the received image to make it difficult to recognize the background pattern. In the case of the first image, depending on the lighting conditions at the time of shooting, it is conceivable that the background pattern may become faintly visible, which could lead to a decrease in accuracy in the forgery detection process in S306. In other words, even if the received image is the original, if the background pattern becomes faintly visible, there is a risk that the system may mistakenly identify the received image as a copied document, as shown in Figure 5(c), and output a result indicating it is a forgery. Therefore, in this step, image processing is performed to make it difficult to recognize the background pattern.

[0094] The process of making it difficult to recognize the background pattern involves adjusting the brightness and luminance of the image, or removing shadows contained within the image, thereby making it difficult to recognize the background pattern within the image.

[0095] It should be noted that the process of making the background pattern difficult to recognize is a different process from the specific image processing used to highlight the background pattern contained in the document. In the second embodiment, an example was described in which the process of making the background pattern difficult to recognize is performed instead of the specific image processing used to highlight the background pattern contained in the document. However, it is sufficient if an image can be obtained in a subsequent process to determine whether the received image is a forgery or not. For example, the specific image processing used to highlight the background pattern contained in the document may be skipped, and the process of making the background pattern difficult to recognize may be performed instead.

[0096] This allows for more accurate output of results in subsequent counterfeit detection processes, ensuring that the results are not unexpected by the user.

[0097] In S604, the CPU 201 acquires a third image that has undergone image processing in S603 to make the background pattern difficult to recognize. Specifically, the third image is stored in a temporary area or memory 202 for result determination. In other words, this is an example of a means for acquiring a first image on which a specific image processing has been performed to make the background pattern difficult to recognize for a given document.

[0098] The processing in S605-S606 is the same as that in S304-S305, so it will be omitted.

[0099] The process in S607 is basically the same as the process in S306. The difference is that while S306 performs a forgery determination process based on the first and second images, S607 performs a forgery determination process based on a third image that has been processed to make the background pattern difficult to recognize and a fourth image (second image) that has been processed to make the background pattern easier to recognize. In other words, this step is an example of a process that determines whether the predetermined document is an original based on the first image, which has not been processed in a way that makes the background pattern stand out, and the second image, in which the first image has been processed in a way that makes the background pattern difficult to recognize.

[0100] Furthermore, if image processing is performed to make it difficult to recognize the background pattern, there is a possibility that the system may not be able to correctly identify a document that has been copied from a document containing the background pattern. In other words, in the case of a pattern like that shown in Figure 5(c), the image processing that makes it difficult to recognize the background pattern may make it difficult to distinguish the word "copy," potentially leading to the system mistakenly identifying it as the original.

[0101] Therefore, the first image, which has not undergone any specific image processing, may be used for the determination. Figure 7 shows an example of the determination result pattern. The forgery determination process is performed based on the first image 402, the third image 701 which has undergone image processing to make the background pattern difficult to recognize, and the fourth image (second image) 702 which has undergone image processing to make the background pattern easier to recognize. If each of the three images follows the pattern in Figure 7(a), it is determined to be an original; otherwise, it is determined to be a forgery.

[0102] In the pattern shown in Figure 7(c), the word "copy" appears in the first image 402, the word "copy" does not appear in the third image 701 due to image processing that makes it difficult to recognize the background pattern, and the word "copy" appears in the fourth image 702 due to image processing that makes it easier to recognize the background pattern.

[0103] In other words, this is an example of a means for determining whether a given document is the original based on a first image in which specific image processing to highlight the background pattern and specific image processing to make the background pattern difficult to recognize have not been performed, a second image in which specific image processing to highlight the background pattern has been performed, and a third image in which specific image processing to make the background pattern difficult to recognize has been performed.

[0104] This makes it possible to correctly identify even images of documents that are copies of the original, without missing any issues. Furthermore, if possible, the identification result may be output based on the third and fourth images, or based on the first, third, and fourth images.

[0105] In the second embodiment, an example was described in which forgery is determined based on three images, but this method is not the only one that can be used. Any other method is acceptable as long as it can output the determination result.

[0106] For example, the processing could be divided into two stages. First, it is determined whether or not a background pattern is visible based on the first image 402, which has not undergone any specific image processing, and the third image 701, which has undergone image processing to make the background pattern difficult to recognize. If a background pattern is visible, it is determined to be a forgery (for example, receiving a copy document like the one in Figure 7(c)). At this stage, the result indicating that it is a forgery may be output, or an error notification such as "Please send the original" may be output.

[0107] If it is determined that the background pattern is not visible, the determination of whether or not it is a forgery is made based on the third image 701 and the fourth image 702, which has been processed to make the background pattern easier to recognize. As shown in Figure 7(a), if the background pattern is visible in the fourth image 702, it is determined that the document contained in the received image is an original.

[0108] The processing in S608-S610 is the same as that in S307-S309, so it will be omitted.

[0109] As described above, according to the second embodiment, it is possible to easily determine whether a document containing a watermark is a forgery or not.

[0110] Furthermore, the watermarks on documents vary from one municipality to another. Therefore, it may be possible to determine whether a document was obtained from the appropriate municipality based on the applicant's information and the image of the received document containing the watermark.

[0111] For example, if a document containing a watermark that can only be issued in one's registered domicile is required, and the applicant's registered domicile is in Shinjuku Ward, Tokyo, then the document containing the watermark must be obtained from the municipality that has jurisdiction over Shinjuku Ward. Furthermore, the document containing the watermark will have a watermark unique to Shinjuku Ward embedded in its background.

[0112] As a method of determination, for example, a trained model is generated by training it with images of documents with a raised background pattern and the name (label) of the issuing authority, such as a local government, associated with those images. For example, an image of a document with a raised background pattern for Shinjuku Ward and the label "Shinjuku Ward" are used to train the model. When an image of a document with a raised background pattern is input to this trained model, it outputs information indicating which local government issued the document. In other words, this is an example of a means of determining the issuing authority of a given document using a trained model that has been trained with images containing a given document with a raised background pattern and a label indicating the issuing authority of that given document with a raised background pattern.

[0113] In this embodiment, the determination of whether or not a document is a forgery is made based on the pattern of presence or absence of a watermark in the first image 402 and the second image 403. In addition, the determination of whether or not the document was obtained from the appropriate local government may also be taken into consideration when outputting the final result.

[0114] For example, after it is determined that the document contained in the received image is the original in the processing according to this embodiment (if it is determined not to be the original, the processing is carried out as described in this embodiment), it is determined whether or not the document was obtained from the appropriate local government. Specifically, it is determined whether or not the document was obtained from the appropriate local government based on the results of the trained model that determines the issuer of the document, the information relating to the applicant, and the document contained in the received image. That is, if these three pieces of information match, it is determined that the document was obtained from the appropriate local government, and if not, error information is output. As error information, for example, information such as "The registered address and the issuer of the sent document do not match" is output.

[0115] In other words, this is an example of a method for determining whether a specified document was issued by the appropriate local government, based on an image containing the specified document, the issuer of the specified document, and information relating to the applicant.

[0116] This makes it easier to verify that documents that are highly credible because they were obtained from a specific municipality (for example, documents that can only be obtained from one's registered domicile) are not documents obtained from other municipalities. For example, if the result is determined solely by the presence or absence of a watermark, if a person other than the applicant submits an application with a document obtained from that person's registered domicile, the result will be judged as an original (because it does not check whether the content of the document matches or not). Therefore, by outputting the final result using information related to the place of issue, it becomes possible to determine whether or not a document is a forgery with greater accuracy.

[0117] In this embodiment, the first and second embodiments have been described as separate processes, but they may also be a series of processes. Specifically, the second embodiment is a method for resolving misjudgments caused by situations where the background pattern becomes faintly visible depending on the lighting conditions during shooting. Therefore, if the received image is determined to be forged as a result of executing the process in the first embodiment, the reliability of that determination is checked using the method of the second embodiment, and the final determination result is output.

[0118] Figure 8 shows an example of a flowchart for a series of processes. Steps S801 to S807 are the same as S301 to S307 and are therefore omitted. Steps S808 to S809 are the same as S603 to S604 and are therefore omitted.

[0119] In S810, the CPU 201 determines whether the received image is a forgery based on a second image that has undergone specific image processing and a third image that has undergone image processing to make the background pattern difficult to recognize. The content of the processing to be executed is the same as in S607. Since this step is performed after the image has been determined to be a forgery in S806, if the determination result is still that it is a forgery, the process proceeds to S811; if it is determined to be an original, the process proceeds to S807.

[0120] Specifically, in S806, the received image was determined to be a forgery based on the combination shown in Figure 5(b) or Figure 5(c). At this time, due to the environment during shooting, the background pattern embedded in the paper may have become faintly visible, potentially causing the combination of the first image (without specific image processing) and the second image (with specific image processing) to be mistakenly identified as the combination shown in Figure 5(c). Therefore, in this step, instead of using the first image (without specific image processing), a third image (with image processing that makes the background pattern difficult to recognize) is used to determine again whether or not it is a forgery.

[0121] Sections S811-S812 are the same as S308-S309, so they are omitted.

[0122] Furthermore, if the first and second embodiments involve separate processes, the system may allow the user to select which process to execute. For example, a screen may be displayed with buttons for executing the first embodiment (first mode) and for executing the process according to the second embodiment (second mode), and the system may execute one of the processes depending on which button is pressed. In addition, on the display screen for the determination result (not shown), not only the first image 402 and the second image 403 may be displayed side by side, and it may be possible to determine which mode the process was executed in.

[0123] This embodiment was explained using an example of determining whether or not documents related to family registers in inheritance procedures are forged (i.e., whether or not they are originals), but it is not limited to this. It may also be applied to a system for determining whether or not other documents containing watermarks, such as resident registration certificates, seal registration certificates, tax certificates, tax payment certificates, graduation certificates, and lease agreements, are forged (i.e., whether or not they are originals).

[0124] The present invention can take the form of, for example, a system, apparatus, method, program, or recording medium. Specifically, it may be applied to a system consisting of multiple devices, or to an apparatus consisting of a single device.

[0125] Furthermore, the various controls described above, which are performed by CPU201, may be performed by a single piece of hardware, or multiple pieces of hardware (for example, multiple processors or circuits) may share the processing to control the entire device.

[0126] Furthermore, although the present invention has been described in detail based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms that do not depart from the spirit of the invention are also included in the present invention. Moreover, each of the embodiments described above is merely one embodiment of the present invention, and it is also possible to combine each embodiment as appropriate.

[0127] (Other embodiments) The present invention can also be realized by performing the following process: supplying software (programs) that realize the functions of the embodiments described above to a system or device via a network or various storage media, and having the computer (or CPU, MPU, etc.) of that system or device read and execute the program code. In this case, the program and the storage medium storing the program constitute the present invention. [Explanation of Symbols]

[0128] 101 Client terminals 102 Servers

Claims

1. Image processing execution means that causes an image processing execution unit to perform specific image processing to make a background pattern stand out, A first acquisition means for acquiring a first image in which specific image processing for making a background pattern appear by the image processing execution unit has not been performed on an image containing a predetermined document, A second acquisition means for acquiring a second image in which a specific image processing is performed by the image processing execution unit to make a background pattern stand out on an image containing the predetermined document, A determination means for determining whether the predetermined document is the original based on the first image and the second image, An information processing system characterized by comprising the following features.

2. The determination means determines whether the predetermined document is the original or not, depending on whether the combination of the first image and the second image satisfies predetermined conditions. The information processing system according to claim 1, characterized by the following:

3. The determination means is, If no background pattern is visible in the first image, but a background pattern is visible in the second image, then the specified document is determined to be the original. The information processing system according to claim 1, characterized by the following:

4. The determination means is, If the first image does not show a background pattern, and the second image is a combination different from the combination in which a background pattern is visible, then it is determined that the specified document is not the original. The information processing system according to claim 1, characterized by the following:

5. The aforementioned different combination is a combination in which no background pattern is visible in the first image and the second image. The information processing system according to claim 4, characterized by the following:

6. The aforementioned different combination is a combination in which a background pattern is visible in both the first image and the second image. The information processing system according to claim 4, characterized by the following:

7. The determination means determines whether the predetermined document is the original or not using a trained model that has been trained on the combination of the first image and the second image and the correct label for that combination. The information processing system according to claim 1, characterized by the following:

8. Display control means for controlling the display of the first image and the second image so that they can be compared. The information processing system according to claim 1, characterized by comprising the following:

9. Display control means for controlling the display of the determination result by the determination means. The information processing system according to claim 1, characterized by comprising the following:

10. The specific image processing required to make the aforementioned background pattern stand out is to perform a binarization process on the image containing the predetermined document. The information processing system according to claim 1, characterized by the following:

11. Receiving means for receiving a single image containing a predetermined document. The information processing system according to claim 1, characterized by comprising the following:

12. The image containing the aforementioned predetermined document is an image captured by an imaging means. The information processing system according to claim 1, characterized by comprising the following:

13. A receiving means that does not receive images stored in an electronic device, but receives images captured by an imaging means. The information processing system according to claim 1, characterized by comprising the following:

14. If the aforementioned determination means determines that the document is not the original, a notification means will notify the document to mail a specified document. The information processing system according to claim 1, characterized by comprising the following:

15. The first acquisition means acquires a first image from the predetermined document, which has been subjected to specific image processing by the image processing execution unit to make the background pattern difficult to recognize. The information processing system according to feature 1.

16. The determination means determines whether the predetermined document is the original based on the first image, which has undergone specific image processing by the image processing execution unit to make the background pattern difficult to recognize, and the second image. The information processing system according to claim 15, characterized in that it is the same as described in claim 15.

17. The determination means determines whether the predetermined document is the original based on a first image in which the image processing execution unit has not performed specific image processing to make the background pattern stand out, but in which the image processing execution unit has performed specific image processing to make the background pattern difficult to recognize, and the second image. The information processing system according to claim 15, characterized in that it is the same as described in claim 15.

18. The determination means determines whether the predetermined document is the original based on a first image in which the image processing execution unit has not performed specific image processing to make the background pattern stand out and specific image processing to make the background pattern difficult to recognize, the second image, and a third image in which the image processing execution unit has performed specific image processing to make the background pattern difficult to recognize. The information processing system according to feature 1.

19. The specific image processing that makes the background pattern difficult to recognize is at least one of the following: adjusting the edge gradient, hue, saturation, brightness, or luminance of the image, or removing shadows in the image. An information processing system according to any one of claims 15 to 18, characterized by the above.

20. The determination means determines whether the prescribed document is a document issued by the appropriate local government, based on an image containing the prescribed document, the issuer of the prescribed document, and information relating to the applicant. The information processing system according to claim 1, characterized by the following:

21. The determination means determines the publisher of the predetermined document using a trained model that has been trained on an image containing the predetermined document with a background pattern raised, and a label indicating the publisher of the predetermined document with the background pattern raised. The information processing system according to claim 20, characterized by the following:

22. The aforementioned specified document is a document that includes a background pattern. An information processing system according to any one of claims 1 to 18, 20 to 21, characterized by the above.

23. The aforementioned background pattern refers to a pattern or string of characters applied to the background of the paper, which becomes visible through scanning or image processing. An information processing system according to any one of claims 1 to 18, 20 to 21, characterized by the above.

24. A method for controlling an information processing system, The image processing execution means of the information processing system performs an image processing execution step in which it causes the image processing execution unit to perform specific image processing to make the background pattern stand out, The first acquisition means of the information processing system acquires a first image in which the image processing execution unit has not performed specific image processing to make a background pattern appear on an image containing a predetermined document, in a first acquisition step, The second acquisition means of the information processing system acquires a second image in which a specific image processing is performed by the image processing execution unit to make a background pattern appear on the image containing the predetermined document. The determination means of the information processing system includes a determination step of determining whether or not the predetermined document is the original based on the first image and the second image, A control method for an information processing system, characterized by comprising the following:

25. A program for causing at least one computer to function as one of the means of the information processing system described in any one of claims 1 to 18, 20 to 21.

Images