Method, apparatus, system and computer program for detecting falsification of document

Abstract

The present invention relates to a method, apparatus, system, and computer program for determining document forgery and alteration, and more specifically, to a method, apparatus, system, and computer program for determining whether a document to be verified is forged or altered through comparison with an original document image. The present invention discloses a method for determining document forgery, characterized in that a document forgery detection system comprises: a step of converting a first document image into a first color to produce a first converted image and increasing the transparency of at least a portion of a second document image to produce a second converted image; and a step of overlaying the second converted image onto the first converted image to produce a first superimposed image in which a changed portion of the second document image is revealed.

Description

Technology Field

[0001] The present invention relates to a method, apparatus, system, and computer program for determining document forgery and alteration, and more specifically, to a method, apparatus, system, and computer program for determining whether a document to be verified is forged or altered through comparison with an original document image. Background Technology

[0002] Recently, the use of electronic documents such as contracts is gradually increasing, but in most cases, they are still created by printing on paper or writing by hand.

[0003] In the case of electronic documents, changes to the content of documents such as contracts can be easily identified by comparing them with the original; however, for documents printed on paper or handwritten, determining whether they have been forged or altered typically required a manual process in which a worker directly verified the authenticity by comparing them with the original.

[0004] In particular, when the volume of documents, such as contracts, increased, not only did the time required for the work increase, but the possibility of worker errors also remained.

[0005] In this regard, attempts are being made to digitize paper documents using optical recognition (OCR) technology and to use this to determine forgery or alteration; however, while such methods may enable the identification of the document's text, it is difficult to determine whether signatures or images have been altered. Furthermore, since the recognition rate may be lower for handwritten documents, there are still significant limitations in accurately determining whether a document has been forged or altered.

[0006] Accordingly, there is a demand for a method to effectively determine whether various types of documents have been forged or altered, but an appropriate solution has not yet been presented. Prior art literature

[0007] Republic of Korea Published Patent No. 10-2009-0054275 (Published May 29, 2009) The problem to be solved

[0008] The present invention was devised to solve the problems of the prior art described above, and aims to provide a method, device, system, and computer program for detecting document forgery that can accurately determine whether various types of documents have been forged or altered by comparing them with the original document.

[0009] In addition, the present invention aims to provide a method, device, system, and computer program for detecting document forgery that can effectively detect forgery even for a large number of documents.

[0010] Other detailed objectives of the present invention will be self-evident and understandable to experts or researchers in the art through the specific details described below. means of solving the problem

[0011] A method for determining document forgery according to one aspect of the present invention for solving the above problem comprises: a document forgery determination system, the step of converting a first document image into a first color to produce a first converted image and increasing the transparency of at least a portion of a second document image to produce a second converted image; and the step of overlaying the second converted image onto the first converted image to produce a first superimposed image in which a changed portion of the second document image is revealed.

[0012] Here, the step of producing the first superimposed image may further include: producing a first superimposed image in which a deleted portion of the second document image is revealed; following the step of producing the first superimposed image, a third transformed image is produced by converting the first document image to a third color and producing a fourth transformed image by increasing the transparency of at least a portion of the second document image; and a step of producing a second superimposed image in which a portion added to the second document image is revealed by overlaying the third transformed image on the fourth transformed image.

[0013] Additionally, the step of producing the first superimposed image may further include: producing a first superimposed image in which a deleted portion of the second document image is revealed; and prior to the step of producing the second transformed image, converting the first document image to a third color to produce a third transformed image and increasing the transparency of at least a portion of the second document image to produce a fourth transformed image; and overlaying the third transformed image onto the fourth transformed image to produce a second superimposed image in which a portion added to the second document image is revealed.

[0014] In addition, a method for determining document forgery according to one embodiment of the present invention may further include the step of aligning the first document image and the second document image.

[0015] At this time, the alignment step may include: a step of calculating feature points of the first document image and the second document image; and a step of aligning the first document image and the second document image using each of the feature points.

[0016] Furthermore, in the step of calculating the above feature points, feature points common to the first document image and the second document image can be calculated.

[0017] In addition, in the step of producing the second converted image, the text of the second document image can be converted to a second color different from the first color and the transparency of the background can be increased to produce the second converted image.

[0018] In addition, in the step of producing the fourth converted image, the text of the first document image can be converted to a fourth color different from the third color and the transparency of the background can be increased to produce the third converted image.

[0019] Additionally, the method may further include a step of inputting one or more of the first superimposed image and the second superimposed image into a pre-trained neural network to determine whether there is falsification in the second document image.

[0020] At this time, the neural network can be trained to determine whether the second document image is tampered with based on the area marked with the first color in the first superimposed image.

[0021] In addition, the neural network can be trained to determine whether the second document image is tampered with based on the area marked with the fourth color in the second superimposed image.

[0022] In addition, the first color and the third color are red-based colors, and the second color and the fourth color are gray-based colors, so that the neural network can identify the area marked with the red-based colors and determine whether the second document image has been tampered with based on this.

[0023] In addition, it may further include the step of identifying and displaying a forgery area of ​​the second document image based on a Class Activation Map (CAM) calculated using the neural network.

[0024] In addition, a computer program according to another aspect of the present invention may be a computer program stored on a computer-readable medium for executing each step of the previously described method for determining document forgery on a computer.

[0025] In addition, a document forgery detection device according to another aspect of the present invention is characterized by comprising: a first and second converted image calculation unit that converts a first document image into a first color to produce a first converted image and increases the transparency of at least a portion of a second document image to produce a second converted image; and a first superimposed image calculation unit that overlays the second converted image onto the first converted image to produce a first superimposed image in which a changed portion of the second document image is revealed. Effects of the invention

[0026] In a method, device, system, and computer program for determining document forgery according to one embodiment of the present invention, it is possible to accurately determine whether various types of documents are forged or altered by comparing them with the original document.

[0027] In addition, the method, device, system, and computer program for determining document forgery according to one embodiment of the present invention can effectively determine forgery even for a large number of documents. Brief explanation of the drawing

[0028] The accompanying drawings, which are included as part of the detailed description to aid in understanding the present invention, provide embodiments of the present invention and explain the technical concept of the present invention together with the detailed description. Figure 1 is a diagram illustrating the configuration of a document forgery detection system according to the prior art. FIG. 2 is a diagram illustrating a flowchart of a method for determining document forgery according to an embodiment of the present invention. FIGS. 3 and 4 are drawings illustrating a process for calculating a deleted area in a method for determining document forgery according to an embodiment of the present invention. FIG. 5 is a diagram illustrating a specific flowchart for the step of aligning document images in a method for determining document forgery according to an embodiment of the present invention. FIGS. 6 and 7 are drawings illustrating a process for calculating an added area in a method for determining document forgery according to an embodiment of the present invention. FIGS. 8 to 10 are drawings illustrating forgery detection using a neural network in a document forgery detection method according to an embodiment of the present invention. FIG. 11 is a diagram illustrating the configuration of a document forgery detection device according to one embodiment of the present invention. Specific details for implementing the invention

[0029] It should be noted that the technical terms used in this invention are used merely to describe specific embodiments and are not intended to limit the scope of the invention. Furthermore, unless specifically defined otherwise in this invention, the technical terms used in this invention should be interpreted in the sense generally understood by those skilled in the art to which this invention pertains, and should not be interpreted in an overly broad or overly narrow sense. Additionally, if a technical term used in this invention is an incorrect technical term that fails to accurately express the concept of this invention, it should be replaced with a technical term that can be correctly understood by a person skilled in the art. Moreover, general terms used in this invention should be interpreted according to their prior definitions or the context, and should not be interpreted in an overly narrow sense.

[0030] Furthermore, singular expressions used in the present invention include plural expressions unless the context clearly indicates otherwise. In the present invention, terms such as "composed of" or "comprising" should not be interpreted as necessarily including all of the various components or steps described in the invention, and should be interpreted as meaning that some of the components or steps may not be included, or that additional components or steps may be included.

[0031] Additionally, terms including ordinal numbers, such as first, second, etc., used in the present invention may be used to describe components, but the components shall not be limited by such terms. The terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component.

[0032] Hereinafter, embodiments according to the present invention will be described in detail with reference to the attached drawings. Identical or similar components are given the same reference numeral regardless of the drawing symbols, and redundant descriptions thereof will be omitted.

[0033] Furthermore, in describing the present invention, detailed descriptions of related prior art are omitted if it is determined that such descriptions could obscure the essence of the invention. Additionally, it should be noted that the attached drawings are intended merely to facilitate an understanding of the technical concept of the present invention and should not be interpreted as limiting the technical concept of the invention.

[0034] Hereinafter, exemplary embodiments of a method, apparatus, system, and computer program for determining document forgery according to the present invention will be described in detail with reference to the attached drawings.

[0035] First, FIG. 1 shows a configuration diagram of a document forgery detection system (100) according to an embodiment of the present invention. As can be seen in FIG. 1, the document forgery detection system (100) according to an embodiment of the present invention may be configured to include an optical document input device (110) that generates a document image for a given document and a document forgery detection device (120) that determines whether the document is forged or altered.

[0036] At this time, as the optical document input device (110), a portable terminal equipped with a camera capable of taking pictures of documents, such as a smartphone, tablet PC, PDA, or mobile phone, may be used, and a scanner capable of scanning the document may also be used, but the present invention is not necessarily limited thereto, and various other devices capable of generating a document image through taking pictures or scanning of a printed document may also be used.

[0037] In addition, FIG. 1 illustrates that an optical document input device (110) is included in a document forgery detection system (100) according to one embodiment of the present invention, but the present invention is not necessarily limited thereto, and in a document forgery detection system (100) according to one embodiment of the present invention, a document image generated in advance by the optical document input device (110) may be stored in a database, etc., and the document image stored in the database, etc. may be used to determine whether the document is forged or altered.

[0038] At this time, the document forgery detection device (120) may be implemented using one or more servers, but the present invention is not necessarily limited thereto and may also be implemented using various devices capable of information processing, such as a dedicated device or a cloud system in which multiple information processing devices are linked.

[0039] In addition, the communication network (130) connecting the optical document input device (110) and the document forgery detection device (120) in FIG. 1 may include wired networks and wireless networks, and specifically, may include various communication networks such as a Local Area Network (LAN), a Metropolitan Area Network (MAN), and a Wide Area Network (WAN). In addition, the network (130) may include the known World Wide Web (WWW). However, the communication network (130) according to the present invention is not limited to the networks listed above, and may include at least a known wireless data network, a known telephone network, or a known wired / wireless television network.

[0040] However, in the present invention, the document forgery detection device (120) is not necessarily required to receive a predetermined input information from the optical document input device (110) through the communication network (130). It is also possible to implement the device in various forms, such as receiving input information directly without passing through the communication network (130), or furthermore, having the document forgery detection device (120) integrated with the optical document input device (110).

[0041] In addition, the optical document input device (110) and the communication network (130) are not necessarily required to be provided in the document forgery detection system (100) according to one embodiment of the present invention, and the document forgery detection device (120) may independently determine whether the document is forged or altered by using a document image, etc., that has been previously generated and stored.

[0042] Additionally, FIG. 2 illustrates a flowchart of a method for determining document forgery according to an embodiment of the present invention. As can be seen in FIG. 2, the method for determining document forgery according to an embodiment of the present invention may include a step (S110) in which a document forgery detection system (100) converts a first document image (21) into a first color to produce a first converted image (31) and increases the transparency of at least a portion of a second document image (22) to produce a second converted image (32), and a step (S120) in which the second converted image (32) is overlaid on the first converted image (31) to produce a first superimposed image (41) in which the changed portion of the second document image (22) is revealed.

[0043] At this time, in a method for determining document forgery according to one embodiment of the present invention, the step of producing the first superimposed image (S120) produces a first superimposed image (41) in which a deleted part of the second document image (22) is revealed, and following the step of producing the first superimposed image (41) (S120), the method may further include a step of producing a third converted image (33) by converting the first document image (21) to a third color, and a step of producing a fourth converted image (34) by increasing the transparency of at least a part of the second document image (22) (not shown), and a step of producing a second superimposed image (42) in which a part added to the second document image (22) is revealed by overlaying the third converted image (33) on the fourth converted image (34) (not shown).

[0044] In addition, in a method for determining document forgery according to an embodiment of the present invention, the step (S120) of producing the first superimposed image (41) produces the first superimposed image (41) in which the deleted portion of the second document image (22) is revealed, and prior to the step (S110) of producing the second converted image (32), the method may further include the step (not shown) of producing the third converted image (33) by converting the first document image (21) to a third color and increasing the transparency of at least a portion of the second document image (22) to produce the fourth converted image (34), and the step (not shown) of producing the second superimposed image (42) in which the deleted portion of the second document image (22) is revealed by overlaying the third converted image (33) on the fourth converted image (34).

[0045] Below, with reference to FIG. 2, a method for determining document forgery according to an embodiment of the present invention is examined in detail, divided into each step.

[0046] First, in the above S110 step, the document forgery detection system (100) converts the first document image (21) into a first color to produce a first converted image (31), and increases the transparency of at least a portion of the second document image (22) to produce a second converted image (32).

[0047] As a more specific example, a method for determining document forgery according to an embodiment of the present invention is described by citing a case in which the authenticity of a document is determined by comparing an original contract document with a sealed copy of the contract. However, the present invention is not limited thereto.

[0048] At this time, in step S110, as can be seen in FIG. 3, a first converted image (31) and a second converted image (32) are produced from a first document image (21) of the original contract document and a second document image (22) of the contract sealing document, respectively.

[0049] However, since the size, tilt, etc. of the image may change during the process of generating the first document image (21) or the second document image (22) using the optical document input device (110), etc., in the method for determining document forgery according to one embodiment of the present invention, a step of aligning the size, tilt, etc. of the first document image (21) and the second document image (22) may be performed.

[0050] More specifically, the step of aligning the first document image (21) and the second document image (22) can be performed by calculating the feature points of the first document image (21) and the second document image (22) as can be seen in FIG. 5 (S151) and aligning the first document image (21) and the second document image (22) using each of the feature points (S152).

[0051] At this time, in the step (S151) of calculating the above feature points, a feature point common to the first document image (21) and the second document image (22) can be calculated.

[0052] More specifically, as can be seen in FIG. 3(b), in the step (S151) of calculating the feature points, common feature points for the first document image (21) and the second document image (22) can be detected using a SURF (Speeded Up Robust Features) algorithm. However, this is an embodiment of the present invention and the present invention is not limited thereto.

[0053] Next, in the alignment step (S152), as can be seen in FIG. 3 (c), the size and angle of the first document image (21) and the second document image (22) can be adjusted based on common feature points detected using a feature matching technique such as OpenCV (Open Source Computer Vision). However, this is an embodiment of the present invention and the present invention is not limited thereto.

[0054] Accordingly, in a method for determining document forgery according to one embodiment of the present invention, with respect to the first document image (21) and the second document image (22) with the size and tilt, etc. aligned as can be seen in (d) of FIG. 3, the first document image (21) is converted to a first color to produce a first converted image (31), and the transparency of at least a part of the second document image (22) is increased to produce a second converted image (32).

[0055] Furthermore, the text of the second image (32) may be converted to a second color different from the first color.

[0056] At this time, it is preferable to select the first color and the second color as colors that can clearly distinguish the difference between the first document image (21) and the second document image (22).

[0057] For example, as can be seen in FIG. 3 (d), the first color may be a red-based color and the second color may be a gray-based color. However, this is an embodiment of the present invention and the present invention is not limited thereto.

[0058] Next, in step S120, the second transformed image (32) is overlaid on the first transformed image (31) to produce a first superimposed image (41) in which the parts changed in the second document image (22) are revealed.

[0059] For a more specific example, the second converted image (32) converted to the second color is overlaid on top of the first converted image (31) converted to the first color (Fig. 4(a)), thereby producing a first superimposed image (41) in which the changed part of the second document image (22) is revealed (Fig. 4(b)).

[0060] More specifically, as can be seen in FIG. 4 (c), when there is an area that exists in the first document image (21) but is deleted in the second document image (22), the area of ​​the first document image (21) corresponding to the deleted area is clearly revealed in the first color (e.g., red), so that the deleted part in the second document image (22) can be effectively detected using the first superimposed image (41).

[0061] Furthermore, in the step S110 above, when performing the second converted image (32), the second converted image (32) is produced by increasing the transparency of the background while converting the text in the second document image (22) to a second color, so that when the first converted image (31) and the second converted image (32) are overlaid to form the first superimposed image (41), the deleted part in the second document image (22) can be detected more effectively.

[0062] Accordingly, in a method for determining document forgery according to one embodiment of the present invention, as previously described, the first superimposed image (41) is generated to effectively detect the changed part in the second document image (22).

[0063] Furthermore, in a method for determining document forgery according to an embodiment of the present invention, the step (S120) of producing the first superimposed image (41) produces a first superimposed image (41) in which a deleted portion of the second document image (22) is revealed, and following the step (S120) of producing the first superimposed image (41), the method may further include a step (not shown) of converting the first document image (21) to a third color to produce a third converted image (33), increasing the transparency of at least a portion of the second document image (22) to produce a fourth converted image (34), and a step (not shown) of overlaying the third converted image (33) on the fourth converted image (34) to produce a second superimposed image (42) in which a portion added to the second document image (22) is revealed.

[0064] In addition, in a method for determining document forgery according to an embodiment of the present invention, the step (S120) of producing the first superimposed image (41) produces the first superimposed image (41) in which the deleted portion of the second document image (22) is revealed, and prior to the step (S110) of producing the second converted image (32), the method may further include the step (not shown) of producing the third converted image (33) by converting the first document image (21) to a third color and increasing the transparency of at least a portion of the second document image (22) to produce the fourth converted image (34), and the step (not shown) of producing the second superimposed image (42) in which the deleted portion of the second document image (22) is revealed by overlaying the third converted image (33) on the fourth converted image (34).

[0065] More specifically, in step S130, the first document image (21) is converted to a third color to produce a third converted image (33), and the second document image (22) is converted to a fourth color different from the third color to produce a fourth converted image (34).

[0066] At this time, the third color may be the same color as the first color (e.g., red), and the fourth color may be the same color as the second color (e.g., gray), but the present invention is not necessarily limited thereto.

[0067] More specifically, in the step of producing the fourth converted image (34), as can be seen in FIG. 6, the third converted image (33) and the fourth converted image (34) are produced from the first document image (21) and the second document image (22), respectively.

[0068] At this time, as previously examined, a step of aligning the first document image (21) and the second document image (22) by adjusting their size and tilt can be performed.

[0069] Accordingly, as can be seen in FIG. 6(b), common feature points for the first document image (21) and the second document image (22) are detected using the SURF (Speeded Up Robust Features) algorithm, and then, as can be seen in FIG. 6(c), the size and angle of the first document image (21) and the second document image (22) can be adjusted based on the detected common feature points using the feature matching technique of OpenCV (Open Source Computer Vision), etc. However, this is an embodiment of the present invention and the present invention is not limited thereto.

[0070] Accordingly, in a method for determining document forgery according to one embodiment of the present invention, as can be seen in (d) of FIG. 6, regarding the first document image (21) and the second document image (22) with the size and tilt, etc. aligned, the first document image (21) is converted to a third color to produce a third converted image (33), and the second document image (22) is converted to a fourth color different from the third color to produce a fourth converted image (34).

[0071] Next, in the step of producing the second superimposed image (42), the third transformed image (33) is overlaid on the fourth transformed image (34) to produce a second superimposed image (42) in which the added part of the second document image (22) is revealed.

[0072] For a more specific example, the third converted image (33) converted to the third color is overlaid on top of the fourth converted image (34) converted to the fourth color (Fig. 7(a)), thereby producing a second superimposed image (42) in which the added part of the second document image (22) is revealed (Fig. 7(b)).

[0073] Accordingly, as can be seen in Fig. 7 (c), if there is an area added in the second document image (22) that does not exist in the first document image (21), the added area is clearly revealed in the fourth color (e.g., red), so that the added part in the second document image (22) can be effectively detected using the second superimposed image (42).

[0074] Furthermore, in producing the first converted image (31), the transparency of the background in the first document image (21) is increased to produce the first converted image (31), so that when the first converted image (31) and the second converted image (32) are overlaid to form the second superimposed image (42), the added part in the second document image (22) can be detected more effectively.

[0075] Accordingly, in a method for determining document forgery according to one embodiment of the present invention, as previously described, the second superimposed image (42) is generated to effectively detect the added part in the second document image (22).

[0076] Furthermore, a method for determining document forgery according to one embodiment of the present invention may further include a step (not shown) of inputting one or more of the first superimposed image (41) and the second superimposed image (42) into a pre-trained neural network to determine whether there is forgery in the second document image (22).

[0077] At this time, the neural network may be a neural network trained to determine whether the second document image (22) is tampered with based on the area marked with the first color in the first superimposed image (41).

[0078] More specifically, in a method for determining document forgery according to one embodiment of the present invention, as illustrated in FIG. 8, a Convolutional Neural Network (CNN) can be used to determine whether the second document image (22) is forged or tampered with. However, this is one embodiment of the present invention and the present invention is not necessarily limited thereto.

[0079] In addition, in a method for determining document forgery according to one embodiment of the present invention, the neural network may be a neural network trained to determine whether the second document image (22) is forged based on the area marked with the fourth color in the second superimposed image (42).

[0080] For a more specific example, in a method for determining document forgery according to one embodiment of the present invention, the first color and the third color are colors of the red family, and the second color and the fourth color are colors of the gray family, so that the neural network can identify the area marked with the red family color and determine whether the second document image (22) is forged based on this.

[0081] Accordingly, in a method for determining document forgery according to one embodiment of the present invention, as can be seen in FIG. 9, for a plurality of documents, images of the original document and the document to be verified are generated as a first superimposed image (41) and a second superimposed image (42) as previously described (Fig. 9 (a)), and by inputting these into a pre-trained neural network to determine forgery (Fig. 9 (b)), it becomes possible to quickly and accurately detect forged documents even for a plurality of documents (Fig. 9 (c)).

[0082] Furthermore, a method for determining document forgery according to one embodiment of the present invention may further include a step (not shown) of identifying and displaying a forgery area of ​​the second document image (22) based on a Class Activation Map (CAM) calculated using the neural network.

[0083] More specifically, in a method for determining document forgery according to one embodiment of the present invention, as can be seen in FIG. 10, a Class Activation Map (CAM) is calculated using weights between a Convolution Layer and a target class, and the region identified as a region of interest in a CNN neural network model can be represented in the form of a heat map (Fig. 10 (a)).

[0084] Accordingly, in a method for determining document forgery according to one embodiment of the present invention, the forgery area of ​​the second document image (22) is identified and displayed based on the Class Activation Map (CAM) calculated using the neural network ((b) of FIG. 10), thereby allowing the administrator to more easily verify the forgery area.

[0085] In addition, a computer program according to another aspect of the present invention is characterized as being a computer program stored on a computer-readable medium to execute each step of the document forgery detection method described above on a computer. The computer program may be a computer program containing machine code generated by a compiler, as well as a computer program containing high-level language code that can be executed on a computer using an interpreter, etc. In this case, the computer is not limited to personal computers (PCs) or laptop computers, but includes any information processing device equipped with a central processing unit (CPU) capable of executing a computer program, such as a server, smartphone, tablet PC, PDA, or mobile phone.

[0086] In addition, the medium may continuously store a program executable by a computer, or temporarily store it for execution or download. Furthermore, the medium may be various recording or storage means in the form of a single or multiple hardware components, and is not limited to a medium directly connected to a computer system but may exist distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media; and media configured to store program instructions, such as ROM, RAM, and flash memory. Additionally, other examples of the medium may include recording or storage media managed by app stores that distribute applications or sites and servers that supply or distribute various other software.

[0087] In addition, FIG. 11 illustrates a configuration diagram of a document forgery detection device (120) according to one embodiment of the present invention.

[0088] As can be seen in FIG. 11, a document forgery detection device (120) according to one embodiment of the present invention may be configured to include a first and second converted image output unit (121) and a first superimposed image output unit (122).

[0089] Furthermore, the document forgery detection device (120) according to one embodiment of the present invention may further include a third and fourth converted image output unit (not shown) and a second superimposed image output unit (not shown).

[0090] Next, the document forgery detection device (120) according to one embodiment of the present invention is examined below, divided by each component. Since more detailed information regarding the document forgery detection device (120) according to one embodiment of the present invention can be inferred from the description of the document forgery detection method according to one embodiment of the present invention described above, more detailed information below is omitted.

[0091] First, in the first and second conversion image output unit (121), the first document image (21) is converted to a first color to produce a first conversion image (31), and the second document image (22) is converted to a second color different from the first color to produce a second conversion image (22).

[0092] In addition, the first overlay image generation unit (122) generates a first overlay image (41) in which the deleted portion of the second document image (22) is revealed by overlaying the second transformed image (32) onto the first transformed image (31).

[0093] In addition, the third and fourth converted image output unit (not shown) converts the first document image (21) into a third color to produce a third converted image (33), and converts the second document image (22) into a fourth color different from the third color to produce a fourth converted image (34).

[0094] Finally, the second superimposed image output unit (not shown) overlays the third transformed image (33) onto the fourth transformed image (34) to produce a second superimposed image (42) in which the part added to the second document image (22) is revealed.

[0095] In addition, the document forgery detection device (120) according to one embodiment of the present invention is not required to receive a predetermined input information from the optical document input device (110) through the communication network (130), and may receive the input information directly without passing through the communication network (130).

[0096] Furthermore, the document forgery detection device (120) according to one embodiment of the present invention does not necessarily have to be configured as a separate device separated from the optical document input device (110), and the document forgery detection device (120) can be implemented in various forms, such as being combined with the optical document input device (110) to form an integrated unit.

[0097] Accordingly, in the method, device, system, and computer program for determining document forgery according to one embodiment of the present invention, it is possible to accurately determine whether various types of documents are forged or altered by comparing them with the original document.

[0098] In addition, the method, device, system, and computer program for determining document forgery according to one embodiment of the present invention can effectively determine forgery even for a large number of documents.

[0099] The embodiments and drawings described herein are merely illustrative and do not limit the scope of the invention in any way. Furthermore, the connections of lines or connecting members between the components depicted in the drawings are illustrative of functional connections and / or physical or circuit connections, and may be replaced or additionally represented as various functional connections, physical connections, or circuit connections in the actual device. Additionally, unless specifically stated with terms such as "essential" or "importantly," a component may not be strictly necessary for the application of the invention.

[0100] In the specification of the present invention (particularly in the claims), the use of the term "the above" and similar descriptive terms may be in both singular and plural. Furthermore, where a range is described in the present invention, it is implied to include inventions applying individual values ​​belonging to said range (unless otherwise stated), and is equivalent to describing each individual value constituting said range in the detailed description of the invention. Additionally, the steps presented in the method inventions of the present invention are not intended to impose a constraint on their sequential order, and the order may be appropriately changed as necessary, unless a specific step must necessarily precede another step depending on the nature of each process. The use of all examples or exemplary terms (e.g., etc.) in the present invention is merely for the purpose of describing the present invention in detail, and the scope of the present invention is not limited by such examples or exemplary terms unless limited by the claims. Furthermore, a person skilled in the art will understand that various modifications, combinations, and changes may be made according to design conditions and elements within the scope of the claims or equivalents. Explanation of the symbols

[0101] 21 : Document 1 Image 22 : Second document image 31 : First transformed image 32 : Second transformed image 33 : Third transformation image 34 : 4th transformation image 41 : First superimposed image 42 : Second superimposed image 100: Document Forgery Detection System 110: Optical document input device 120 : Document forgery detection device 121 : 1st and 2nd converted image output units 122 : First superimposed image calculation unit 123 : 3rd, 4th converted image output unit 124 : First superimposed image calculation unit 130 : Communication Network

Claims

Claim 1 A method for determining document forgery, comprising: a step of aligning a first document image and a second document image to have the same size and slope based on common feature points detected in the first document image and the second document image, converting the first document image to a first color to produce a first converted image, and increasing the transparency of at least a portion of the second document image to produce a second converted image; and a step of overlaying the second converted image onto the first converted image to produce a first superimposed image in which the first color of the area corresponding to the changed portion of the second document image is revealed by transmitting the second converted image, and the same portion is obscured by the second converted image so that the first color is not displayed. Claim 2 A method for determining document forgery according to claim 1, further comprising: a step of generating a first superimposed image, wherein a first superimposed image is generated in which a deleted portion of the second document image is revealed; a step of generating a third transformed image by converting the first document image to a third color and generating a fourth transformed image by increasing the transparency of at least a portion of the second document image; and a step of generating a second superimposed image in which a portion added to the second document image is revealed by overlaying the third transformed image onto the fourth transformed image. Claim 3 A method for determining document forgery according to claim 1, further comprising: a step of generating a first superimposed image in which a deleted portion of the second document image is revealed; a step of generating a third transformed image by converting the first document image to a third color prior to the step of generating the second transformed image, and generating a fourth transformed image by increasing the transparency of at least a portion of the second document image; and a step of generating a second superimposed image in which a portion added to the second document image is revealed by overlaying the third transformed image on the fourth transformed image. Claim 4 A method for determining document forgery, characterized by further including the step of aligning the first document image and the second document image in claim 1. Claim 5 A method for determining document forgery according to claim 4, wherein the alignment step comprises: a step of calculating feature points of the first document image and the second document image; and a step of aligning the first document image and the second document image using each of the feature points. Claim 6 A method for determining document forgery according to claim 5, wherein in the step of calculating the feature points, a feature point common to the first document image and the second document image is calculated. Claim 7 A method for determining document forgery according to claim 1, wherein in the step of producing the second converted image, the text of the second document image is converted to a second color different from the first color and the transparency of the background is increased to produce the second converted image. Claim 8 A method for determining document forgery, characterized in that, in the step of producing the fourth converted image in claim 2 or 3, the text of the first document image is converted to a fourth color different from the third color and the transparency of the background is increased to produce the third converted image. Claim 9 A method for determining document forgery, characterized by further including the step of inputting one or more of the first superimposed image and the second superimposed image into a pre-trained neural network to determine whether there is forgery in the second document image, in accordance with claim 2 or 3. Claim 10 A method for determining document forgery according to claim 9, characterized in that the neural network is trained to determine whether the second document image is forged or tampered with based on the area marked with the first color in the first superimposed image. Claim 11 A method for determining document forgery according to claim 9, characterized in that the neural network is trained to determine whether the second document image is forged based on an area marked with a fourth color different from the third color in the second superimposed image. Claim 12 A method for determining document forgery according to claim 9, wherein the first color and the third color are colors of the red family, and the second color different from the first color and the fourth color different from the third color are colors of the gray family, and the neural network identifies the area marked with the red family colors and determines whether the second document image is forged or tampered with based thereon. Claim 13 A method for determining document forgery according to claim 9, further comprising the step of identifying and displaying a forgery area of ​​the second document image based on a Class Activation Map (CAM) calculated using the neural network. Claim 14 A computer program stored on a computer-readable medium for executing each step of the method for determining document forgery described in any one of paragraphs 1 through 7 on a computer. Claim 15 A document forgery detection device characterized by comprising: a first and second converted image calculation unit that aligns a first document image and a second document image to have the same size and slope based on common feature points detected in the first document image and the second document image, converts the first document image to a first color to produce a first converted image, and increases the transparency of at least a portion of the second document image to produce a second converted image; and a first superimposed image calculation unit that overlays the second converted image onto the first converted image to produce a first superimposed image in which the first color of the area corresponding to the changed part of the second document image is revealed by transmitting the second converted image, and the same part is obscured by the second converted image so that the first color is not displayed.

Images