Character matching device, program, and enclosure confirmation system

Abstract

To provide a character collation device, a program and an enclosure confirmation system for devising compensation for the accuracy of an OCR.SOLUTION: An enclosure confirmation device 1 includes a character recognition unit 15 for performing character recognition processing on an image obtained by photographing an enclosure, a character collation unit 18 for comparing and collating each character string corresponding to a plurality of enclosures, an image similarity calculation unit 20 for comparing images of an area to be a collation object between documents to calculate an image similarity, a character determination unit 21 for determining whether identification character information is coincident on the basis of a collation result by the character collation unit 18 and the image similarity calculated by the image similarity calculation unit 20, and a confirmation result output unit 23 for outputting a determination result to a monitor 4.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a character matching device, a program, and an enclosure verification system. [Background technology]

[0002] Traditionally, the task of enclosing multiple documents, such as notices and statements to be sent to the same person, into an envelope is common. Since documents contain information that can identify an individual, such as an address or name, accidentally enclosing documents with a different destination can lead to the leakage of personal information. Therefore, when enclosing documents, it is necessary to confirm that the documents being enclosed are all for the same person. There are various initiatives to assist in this confirmation process. For example, some documents enclosed in the same envelope are printed with an optically readable code, such as a two-dimensional code or barcode, to confirm that they are addressed to the same person. However, there are cases where printing an optical code on a form is not possible due to reasons such as placing importance on design.

[0003] Therefore, for example, "a document inspection device is provided which is at least composed of a camera device for photographing a document, a document inspection device for inspecting documents to be enclosed in the same envelope, and a data storage device having an inspection history database for storing the inspection history of the documents, the document inspection device is provided with a character recognition unit for character recognition of characters included in the image, and a document inspection unit for inspecting documents to be enclosed in the same envelope, and when an image of a document number composed at least of a serial number and a branch number photographed by a user is output from the camera device, the document inspection unit performs an image acquisition process for storing the image of the document number output from the camera device in a memory, and a data storage device having an inspection history database for storing the inspection history of the documents to be enclosed in the same envelope. The document inspection system is characterized by the following: when it is indicated that photographing of the document numbers for all documents to be inserted has been completed, all images of the document numbers stored in the memory are subjected to character recognition by the character recognition unit, and a verification test is performed to confirm whether the serial numbers of the document numbers for all documents to be enclosed in the same envelope match; and when the document inspection process is completed, a history storage process is performed to store the images of the document numbers stored in the memory in the inspection history database of the data storage device while associating them with the results of the verification test (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-4365 Summary of the Invention [Problem to be solved by the invention]

[0005] Optical Character Recognition (OCR), for example, is used to recognize characters contained in an image, including the one described in Patent Document 1. However, there is a limit to the accuracy of OCR, and there is a risk of misrecognition. In particular, there is a risk of misrecognition when there is a frame around the character to be recognized or when there are other characters. There are also cases where hair or wrinkles in the paper are mistakenly recognized as numbers.

[0006] Therefore, an object of the present invention is to provide a character matching device, a program, and an enclosure confirmation system that are devised to compensate for the accuracy of OCR. [Means for solving the problem]

[0007] The present invention solves the above problems by the following means. A first invention is a character matching device that matches multiple types of documents having the same identification character information, comprising: recognition processing means that performs character recognition processing on images of the documents; character matching means that compares and matches each character string corresponding to the multiple documents; image similarity calculation means that compares images of areas to be matched between the documents and calculates image similarity; determination means that determines whether the identification character information matches based on the matching result by the character matching means and the image similarity calculated by the image similarity calculation means; and determination result output means that outputs the determination result by the determination means to a display unit. A second invention is a character matching device according to the first invention, wherein the image similarity calculation means compares images of each character in the area to be matched between documents to calculate image similarity. A third invention is a character matching device according to the first or second invention, further comprising an area specification means for specifying an area to be matched from the character string obtained by the recognition processing means, and the character matching means compares and collates each character string corresponding to the area to be matched specified by the area specification means. A fourth invention is a character matching device according to the first or second invention, wherein the area to be matched is stored for each of the documents, and the character matching device further comprises a type recognition means for analyzing an image of the document to recognize the type of the document, and the character matching means compares and matches each character string corresponding to the area to be matched for each of the documents based on the type of the document recognized by the type recognition means. A fifth invention is a character matching device according to any one of the first to fourth inventions, further comprising: a preprocessing means for preprocessing an image of the document; and a post-preprocessing recognition processing means for performing character recognition processing on the image preprocessed by the preprocessing means, wherein the character matching means performs matching using character strings corresponding to the plurality of documents obtained by the recognition processing means and character strings corresponding to the plurality of documents obtained by the post-preprocessing recognition processing means. A sixth aspect of the present invention is the character matching device according to the fifth aspect of the present invention, wherein the preprocessing means performs a plurality of types of preprocessing on an image obtained by photographing the document. A seventh invention is a character matching device according to any one of the first to sixth inventions, further comprising an output string acquisition means for acquiring an output string, and the determination result output means further outputs the output string acquired by the output string acquisition means and an image of each area to be matched. An eighth aspect of the present invention is a program for causing a computer to function as any one of the character matching devices according to the first to seventh aspects of the present invention. A ninth invention is an enclosure confirmation system including a character matching device of any one of the first to seventh inventions, comprising a placement stage on which a plurality of the enclosure documents can be placed side by side, a photographing unit capable of photographing the placement stage, an image acquisition means for acquiring, via the photographing unit, a captured image including an image of the plurality of documents placed on the placement stage, and an image extraction means for extracting an image of each document from the captured image acquired by the image acquisition means. The tenth invention is an enclosure confirmation system according to the ninth invention, which comprises a weight detection unit provided in the placement area of ​​each enclosure on the placement stand, a weight memory unit that stores the weight of each type of enclosure, a weight confirmation means that confirms whether the weight of each enclosure detected by the weight detection unit matches the weight of each enclosure stored in the weight memory unit, and an alarm means that issues an alarm when the weight confirmation means confirms that the weights do not match. [Effects of the Invention]

[0008] According to the present invention, it is possible to provide a character matching device, a program, and an enclosure confirmation system that are devised to compensate for the accuracy of OCR. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram showing an overview of an enclosure confirmation system according to an embodiment of the present invention; [Figure 2] 1 is a diagram showing functional blocks of an enclosure checking device according to an embodiment of the present invention; [Figure 3] FIG. 2 is a diagram illustrating an example of a storage unit of the enclosure checking device according to the embodiment. [Figure 4] 6 is a flowchart showing an enclosure checking process in the enclosure checking device according to the present embodiment. [Figure 5] 10 is a flowchart showing a verification area specifying process in the enclosure check device according to the present embodiment. [Figure 6] 10 is a flowchart showing a character matching process in the enclosure checking device according to the embodiment. [Figure 7] FIG. 4 is a diagram showing an example of a screen output to a monitor according to the embodiment. [Figure 8] 5A and 5B are diagrams for explaining the processing in the enclosure checking device according to the embodiment; [Figure 9] FIG. 10 is a diagram showing an example of a determination result screen output to a monitor according to the embodiment. [Figure 10] FIG. 10 is a diagram showing an example of a determination result screen output to a monitor according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, this is merely an example, and the technical scope of the present invention is not limited to this example. (Embodiment) <Enclosure confirmation system 100> FIG. 1 is a diagram showing an outline of an enclosure confirmation system 100 according to this embodiment. The enclosure confirmation system 100 is used, for example, by a contracted service company that has received a request from a business partner in the process of enclosing multiple documents (enclosures) into envelopes to be sent to the business partner's customers, and is used to confirm the enclosing of multiple documents for the same person.

[0011] 1, in the enclosure confirmation system 100, worker P takes one sheet from each of stacks of documents 51 to 54 placed in a document storage area to the right of worker P, starting from the top, and places the documents on a tray 40 in front of worker P in the order in which they are placed in the document storage area. In the example of FIG. 1, stacks of documents 51 to 54, in which four types of documents are stacked in the order in which they were printed, are placed in the document storage area. Here, the documents are, for example, notices, statements, etc. sent to customers by business partners such as financial institutions such as banks and credit card companies, or insurance companies such as life insurance companies and non-life insurance companies. Each document is assigned key information (identification character information) for identifying the customer. The key information is, for example, at least a part of a cash card or credit card number, or an insurance policy number. Furthermore, the documents are printed based on data received from business partners, and the documents in the stacks 51 to 54 are all stacked in the same order. In other words, the order of the documents in each stack 51 to 54 is Customer X, Customer Y, etc.

[0012] The placement table 40 is covered with a single-colored cloth or paper material, for example, green. If the placement table 40 is green, it has the advantage that it serves as a green background during image processing, making it easier to extract an image of an object placed on the placement table 40. Note that the color may be blue instead of green. The placement table 40 is provided with placement areas 41 to 44 on which documents are placed, which are visible to the worker P. The worker P places the documents 51a to 54a in the placement areas 41 to 44 so that each document fits within the placement areas 41 to 44.

[0013] Then, when worker P operates switch 8 provided on platform 40, camera 5 (photographing unit) photographs platform 40. Then, a confirmation result, including the photographed image, as to whether it is of the same person is displayed on monitor 4 (display unit). Switch 8 is an operating member for acquiring images via camera 5. Switch 8 is provided on the front right side of platform 40 as viewed from worker P, and can be operated by worker P's right hand. Note that switch 8 may also be provided in a position where it can be operated by worker P's left hand. Camera 5 is provided in advance, adjusted to a position where it can photograph platform 40.

[0014] If worker P is able to confirm from the confirmation results displayed on monitor 4 that the documents belong to the same person, he or she picks up documents 51a to 54a that were placed on table 40 again, stacks documents 51a to 54a in order, and places the documents in an empty envelope 55 in the envelope holder to the left of worker P. Through this series of operations by worker P, multiple documents 51a to 54a for the same person are placed and sealed in one envelope 55.

[0015] <Enclosure Check Device 1> Next, a description will be given of the functional blocks of the enclosure verification device 1 (character matching device) used in the enclosure verification system 100 to verify whether documents belong to the same person. FIG. 2 is a diagram showing functional blocks of the enclosure checking device 1 according to this embodiment. FIG. 3 is a diagram showing an example of the storage unit 30 of the enclosure checking device 1 according to this embodiment. The enclosure checking device 1 is a device for checking that a plurality of documents 51a to 54a placed on the placing table 40 shown in FIG. 1 are to be sent to the same person. As shown in FIG. 2, the enclosure checking device 1 includes a control unit 10, a storage unit 30, a monitor 4, a camera 5, a speaker 6, weight sensors 71 to 74 (weight detection units), and a switch 8.

[0016] The control unit 10 is a central processing unit (CPU) that controls the entire enclosure checking device 1. The control unit 10 appropriately reads and executes the operating system (OS) and application programs stored in the storage unit 30, thereby cooperating with the above-mentioned hardware and performing various functions. The control unit 10 includes a start acceptance unit 11, an image acquisition unit 12 (image acquisition means), an image cutting unit 13 (image cutting means), a matching area identification unit 14 (type recognition means, area identification means), a character recognition unit 15 (recognition processing means), a preprocessing unit 16 (preprocessing means), a post-preprocessing character recognition unit 17 (post-preprocessing recognition processing means), a character matching unit 18 (character matching means), an output string acquisition unit 19 (output string acquisition means), an image similarity calculation unit 20 (image similarity calculation means), a character determination unit 21 (determination means), a weight confirmation unit 22 (weight confirmation means), and a confirmation result output unit 23 (determination result output means, notification means).

[0017] The start acceptance unit 11 accepts the operation of the switch 8 by the worker P, thereby accepting the start of the process. In response to the reception by the start reception unit 11, the image acquisition unit 12 acquires a photographed image of the mounting table 40 via the camera 5. The image cutting unit 13 cuts out a document image (a photographed image of a document) that is an image of a document portion from the captured image acquired by the image acquisition unit 12. As described above, the placement table 40 can detect the outline of the document, which is the document area, from the green screen background using color information, and cut out the document image. The image cutting unit 13 may also perform keystone correction on the cut-out document image.

[0018] The collation area identification unit 14 recognizes the type of document by analyzing the document image cut out by the image cutout unit 13. The collation area identification unit 14 may recognize the type of each document image by using a technique such as a convolutional neural network (CNN). Furthermore, instead of identifying documents by analyzing the document image, documents may be identified by previously associating the document type with the position of the document image relative to the entire captured image. Then, the matching area specification unit 14 specifies a matching area (area to be matched) from the document image cut out by the image cutout unit 13. The matching area is an area that contains key information for identifying the customer to be matched. For example, the matching area specification unit 14 can specify the matching area by registering the identification area as a coordinate area for each document in advance.

[0019] Character recognition unit 15 performs OCR processing (character recognition processing) on ​​the collation area identified by collation area identification unit 14 to obtain a character string. The preprocessing unit 16 performs preprocessing on the matching region identified by the matching region identification unit 14. The preprocessing unit 16 can perform processing using a variety of different preprocessing techniques. Examples of preprocessing techniques include sharpening, brightness conversion, thinning, and color extraction. Sharpening refers to emphasizing the edges of an image, resulting in a sharper image. Brightness conversion refers to gamma correction, which can emphasize low-brightness areas. Thinning refers to the process of thinning a line in a binarized image so that only one pixel at the center of the line remains, and calculating the center line. Color extraction refers to extracting representative colors with high appearance frequencies, for example, by checking the image histogram. For example, if an image contains red text, color extraction can be used to extract the red text. The preprocessing character recognition unit 17 performs OCR processing on the collation region preprocessed by the preprocessing unit 16, thereby obtaining a character string.

[0020] Character matching unit 18 compares the character strings obtained by character recognition unit 15 with the character strings obtained by preprocessing character recognition unit 17. For example, character matching unit 18 may compare character strings obtained by character recognition unit 15 for the same matching region, and may also compare character strings obtained by preprocessing character recognition unit 17. Furthermore, for example, when multiple preprocessing methods are used, character matching unit 18 may compare multiple character strings obtained by preprocessing character recognition unit 17 for each preprocessing method, or may compare all character strings obtained by preprocessing character recognition unit 17 regardless of the preprocessing method.

[0021] The output character string acquisition unit 19 acquires an output character string from a plurality of character strings. For example, the output character string acquisition unit 19 may acquire, as the output character string, a character string obtained by the character recognition unit 15. Alternatively, the output character string acquisition unit 19 may acquire, as the output character string, a character string with the highest degree of match as a result of collation by the character collation unit 18.

[0022] The image similarity calculation unit 20 compares the images of each character in the matching area identified by the matching area identification unit 14 across multiple documents to calculate the image similarity for each character. Image similarity can be calculated using techniques such as feature point matching using the AKAZE (Accelerated KAZE) algorithm or template matching. All of these techniques are based on the premise that identical characters will have similar image features (structural information), and the same characters will have a high calculated image similarity.

[0023] Character determination unit 21 determines whether or not the key information matches based on the matching result by character matching unit 18. Character determination unit 21 also determines whether or not the key information matches based on the matching result by character matching unit 18 and the image similarity calculated by image similarity calculation unit 20. Character determination unit 21 may determine that the key information matches, for example, when the image similarity calculated by image similarity calculation unit 20 is equal to or greater than a threshold. Here, the threshold may be determined statistically, for example, based on the distribution of image similarities over the past month.

[0024] In the weight confirmation unit 22, weight sensors 71 to 74 detect the weight in response to acceptance by the start acceptance unit 11. Then, the weight confirmation unit 22 confirms whether the detected weight of the document matches the weight of each document stored in the document storage unit 32 (weight storage unit) described later. The confirmation result output unit 23 outputs the determination result by the character determination unit 21 to the monitor 4. The confirmation result output unit 23 may further add the output character string acquired by the output character string acquisition unit 19 and the image of each collation area to the determination result and output it. Furthermore, when the weight confirmation unit 22 confirms that the weights do not match, the confirmation result output unit 23, for example, notifies the user that the weights do not match on the monitor 4. Note that, when the confirmation result output unit 23 confirms that the weights do not match, for example, the confirmation result output unit 23 may notify the user by sounding an alarm from the speaker 6. Details of each process will be described later.

[0025] The storage unit 30 is a storage area such as a hard disk or semiconductor memory element for storing programs, data, etc. required for the control unit 10 to execute various processes. Here, a computer refers to an information processing device equipped with a control unit, a storage device, etc., and the enclosure checking device 1 is an information processing device equipped with a control unit 10, a storage unit 30, etc., and is included in the concept of a computer. The storage unit 30 includes a program storage unit 31, a document storage unit 32, and an enclosing order storage unit 33.

[0026] The program storage unit 31 is a storage area for storing various programs. The program storage unit 31 stores programs (not shown) for performing various functions executed by the control unit 10 of the enclosure checking device 1. The document storage unit 32 is a storage area that stores information about a document, including the document's weight. Fig. 3(A) shows example items in the document storage unit 32. The document storage unit 32 shown in Fig. 3(A) stores a document ID (IDentification), an image, a document name, key area coordinates, and a weight in association with each other. The document ID is identification information that identifies the type of document, and a different ID is assigned to each of document A, document B, document C, . . . The image is an image of the document, and the document name is the name of the document. The key area coordinates are the coordinates of the key information within the document. The weight is the weight per enclosed document, and is expressed in units of mg (milligrams), for example.

[0027] The enclosing order storage unit 33 is a storage area that stores, for each job corresponding to a request from a client, the stacking order of documents to be enclosed and a document ID corresponding to the position on the placing table 40, in association with each other. Fig. 3(B) shows an example of items in the enclosing order storage unit 33. The enclosing order storage unit 33 shown in Fig. 3(B) stores, for each job ID, the stacking order, the location, and the document ID in association with each other. The task ID corresponds to a request from a business partner, and one task ID is assigned to each request. The stacking order is the order in which the enclosed documents are stacked. The location has a name and a position on the table 40. The name is what the KEY is stacked on top of, for example, a document visible through a window in an envelope. SUB1 to SUB3 are names corresponding to the stacking order. The document ID is identification information that identifies the type of document. Additionally, the storage unit 30 may include, for example, a customer information storage unit that stores customer information and task IDs, but detailed description thereof will be omitted.

[0028] 1, the monitor 4 is provided, for example, at the rear side of the table 40 as seen from the worker P, in a position that is easily visible to the worker P who places documents on the table 40. The monitor 4 is a display device that functions as a display unit, and is configured, for example, with a liquid crystal panel or the like. The camera 5 is an imaging device and is provided vertically above the mounting table 40 so that a lens (not shown) faces the mounting table 40, and captures an image of the mounting table 40. The speaker 6 is an audio output device and is provided near the monitor 4, for example. Weight sensors 71 to 74 are sensors that detect weight and are provided on the loading platform 40. Weight sensors 71 to 74 detect the weight of documents placed on them, respectively. Weight sensor 71 is provided, for example, near the center of loading area 41. Like weight sensor 71, weight sensors 72 to 74 are also provided near the centers of loading areas 42 to 44, respectively. The switch 8 is a button that the worker P operates.

[0029] <Processing Description> Next, the process performed by the enclosure checking device 1 will be described. FIG. 4 is a flowchart showing the enclosure checking process in the enclosure checking device 1 according to this embodiment. FIG. 5 is a flowchart showing the verification area identification process in the enclosure checker 1 according to this embodiment. FIG. 6 is a flowchart showing the character matching process in the enclosure check device 1 according to this embodiment. FIG. 7 is a diagram showing an example of a screen 60 output to the monitor 4 according to this embodiment. FIG. 8 is a diagram for explaining the processing in the enclosure checking device 1 according to this embodiment. 9 and 10 are diagrams showing an example of a determination result screen 90 output to the monitor 4 according to this embodiment.

[0030] As a preliminary operation, for example, worker P registers information about the document to be collated in document storage unit 32. Next, worker P places the documents of the various types to be enclosed in one envelope on table 40 in order, as described with reference to FIG.

[0031] In step S (hereinafter referred to as "S") 11 of FIG. 4, the control unit 10 (start reception unit 11) of the enclosure checking device 1 determines whether or not it has received operation of the switch 8. When the worker P places documents corresponding to each area in the placement areas 41 to 44 of the placement table 40 and presses the switch 8, the control unit 10 determines that it has received operation of the switch 8. If it has received operation of the switch 8 (S11: YES), the control unit 10 proceeds to S12. On the other hand, if it has not received operation of the switch 8 (S11: NO), the control unit 10 remains in this process until it receives operation of the switch 8. In S12, the control unit 10 performs a weight detection process. More specifically, in response to receiving a switch operation through the process of S11, the control unit 10 causes the weight sensors 71 to 74 to detect the weights, respectively. Then, the control unit 10 temporarily stores the detected weights in the memory unit 30.

[0032] In S13, the control unit 10 performs a matching area specification process. Here, the collation area specification process will be described with reference to FIG. 5, the control unit 10 (image acquisition unit 12) captures an image of the table 40. In the example of FIG. 1, the captured image includes four different documents 51a to 54a. In S22, the control unit 10 outputs the acquired photographed image to the monitor 4. FIG. 7 is an example of a screen 60 output to the monitor 4. The screen 60 includes document areas 61 to 64 corresponding to the placement areas 41 to 44 shown in FIG. The worker P can confirm by looking at the screen 60 of the monitor 4 that the camera 5 has taken an image as a result of him or her operating the switch 8.

[0033] In S23, the control unit 10 (image cutting unit 13) cuts out a document image from the captured image. The control unit 10 may perform a process of correcting the cut-out document image into a rectangular shape by trapezoidal correction. In S24, the control unit 10 (matching area identification unit 14) identifies a matching area for each cut-out document image. The control unit 10 identifies the position of the matching area for each document by referring to the document storage unit 32, and then identifies the matching area. For example, image 81A shown in FIG. 8(A) is an image in which the matching area of ​​document 51a has been identified, and image 81B shown in FIG. 8(B) is an image in which the matching area of ​​document 52a has been identified. Thereafter, the control unit 10 proceeds to S14 in FIG. 4. In S14 of FIG. 4, the control unit 10 performs character matching processing.

[0034] The character matching process will now be described with reference to FIG. In S31 of FIG. 6, the control unit 10 (character recognition unit 15) performs OCR processing on each collation area to obtain characters included in each collation area. Character string 81X shown in FIG. 8(A) is a character string obtained from image 81A of the collation area. Here, in FIG. 8(A), the character string in image 81A and character string 81X are the same. On the other hand, character string 81Y shown in FIG. 8(B) is a character string obtained from image 81B of the collation area. In FIG. 8(B), the character string in image 81B and character string 81Y are not the same. This is because when OCR processing was performed on image 81B, the line to the left of the number 9 was recognized as the number 1. In this way, OCR processing can result in misrecognition depending on the accuracy of the OCR.

[0035] In S32 of FIG. 6, the control unit 10 (preprocessing unit 16) performs preprocessing on each matching region. Images 82A to 85A shown in Fig. 8(A) are images obtained by performing sharpening, brightness conversion, thinning, and representative color extraction on image 81A, respectively. The same is true for images 82B to 85B shown in Fig. 8(B).

[0036] In S33 of FIG. 6, the control unit 10 (post-preprocessing character recognition unit 17) performs OCR processing on each preprocessed collation area, and obtains the character string included in each collation area. Character strings 82X to 85X shown in Fig. 8(A) are character strings obtained from images 82A to 85A of the matching area, respectively. Character strings 82Y to 85Y shown in Fig. 8(B) are character strings obtained from images 82B to 85B of the matching area, respectively.

[0037] 6, the control unit 10 (character matching unit 18) matches each of the acquired character strings. More specifically, the control unit 10 matches the character strings acquired in the process of S31 for the same matching region. Furthermore, the control unit 10 compares the character strings obtained by the process of S33 with each other, and the control unit 10 (character determination unit 21) determines whether the key information matches. If the result of comparing the character strings obtained in the process of S31 shows that the character strings are the same, the control unit 10 determines that they match, and further compares the character strings obtained in the process of S33 to determine whether they match. On the other hand, if the result of comparing the character strings obtained in the process of S31 shows that the character strings are not the same, the control unit 10 determines that they do not match.

[0038] Here, various methods can be used to compare the multiple character strings obtained by OCR processing after multiple preprocessing steps. One method is to compare OCR results that have undergone the same preprocessing. In this case, the success rate of the comparison for each preprocessing is used as the confidence level, and if the confidence level is 100%, the control unit 10 determines that there is a match. If the confidence level is not 100%, the control unit 10 outputs the judgment result and performs processing such as issuing an alert, which will be described later.

[0039] For example, the control unit 10 checks whether a character string recognized by OCR processing after sharpening of document 51a is identical to a character string recognized by OCR processing after sharpening of document 52a, which is a document by the same person. Similarly, the control unit 10 checks whether a character string recognized by OCR processing after brightness conversion of document 51a is identical to a character string recognized by OCR processing after brightness conversion of document 52a, which is a document by the same person. The control unit 10 then determines the number of matches out of the number of matches as the confidence level. The number of matches, i.e., if there are four types of preprocessing methods and three matches are found to be identical, the confidence level is 3 / 4 (=0.75).

[0040] Another method may be, for example, a simple comparison of the frequency of appearance of each number. The control unit 10, for example, tallies the number of times each number appears for each document. Then, the control unit 10 calculates the difference in the number of times each number appears for each number, and determines the average of the absolute values ​​of the differences as the mismatch rate. If the mismatch rate is 0, the control unit 10 determines that there is a match; if the mismatch rate is other than 0, the control unit 10 performs processing such as issuing an alert when outputting the determination result, which will be described later. The above method is merely an example, and other methods may be used.

[0041] 6, the control unit 10 (output character string acquisition unit 19) acquires an output character string. The control unit 10 can set the output character string to, for example, the character string of the document of KEY acquired in the process of S31. In S36, the control unit 10 (image similarity calculation unit 20) compares the image for each character in each matching region, and calculates the image similarity. In S37, the control unit 10 (character determination unit 21) determines whether the key information matches based on the calculated image similarity and a threshold value. If the image similarity is equal to or greater than the threshold value, the control unit 10 determines that the key information matches. Thereafter, the control unit 10 proceeds to S15 in FIG. 4.

[0042] In S15 of Fig. 4, the control unit 10 (weight confirmation unit 22) performs a weight confirmation process. The control unit 10 confirms whether the weights corresponding to the weight sensors 71 to 74 temporarily stored in the memory unit 30 in the process of S12 match the weights stored in the document memory unit 32. The control unit 10 may determine that the weights match even if there is a slight error in the weights (for example, an error of about 10% from the weight stored in the document memory unit 32). This is because the weight confirmation is mainly used to detect the incorrect placement of multiple sheets. In S16, the control unit 10 (confirmation result output unit 23) outputs the determination result to the monitor 4.

[0043] 9 shows an example of a determination result screen 90 output to the monitor 4. The determination result screen 90 may be output as a pop-up screen superimposed on the screen 60 of FIG. The judgment result screen 90 includes a key information output section 91 , matching area output sections 92 to 95 for each document, and a result output section 96 . The key information output section 91 is an area that outputs an output character string. In the example of Fig. 9, two pieces of key information are extracted, and two output character strings are output. The collation area output units 92 to 95 are areas that output the collation areas of the documents 51a to 54a. The result output section 96 is an area that displays the collation result in an easy-to-understand manner at a glance, and displays the collation result using, for example, "◯", "×", "Δ" or the like. The message output section 97 is an area where a message is output.

[0044] The determination result screen 90 shown in Fig. 9 is an example that is output when the determination result is a match. Therefore, a mark "○" indicating a match is output to the result output unit 96. Here, a match can be, for example, (a) The character string obtained by simply performing OCR processing on the matching area matches, (b) The string obtained by OCR processing after preprocessing matches. (c) The image is judged to match based on the image similarity. (d) weight is consistent; This refers to the case. As shown in the judgment result screen 90 of FIG. 9, when the judgment result is a match, no particular message is output to the message output section 97, and the section remains blank.

[0045] On the other hand, the judgment result screen 90-2 shown in FIG. 10 is an example output when the judgment result is a mismatch. The example in FIG. 10 shows a case where the collation area output unit 93-2 shows a mismatch. The collation area output unit 93-2 has the corresponding column colored to clearly indicate a mismatch. Furthermore, the result output unit 96-2 outputs an "x" mark indicating a mismatch. Here, a mismatch refers to, for example, a case where the character strings acquired by simply performing OCR processing on the collation area do not match, or a case where the weights do not match. A weight mismatch also applies when multiple forms are stacked or when there are not enough documents to be placed.

[0046] The mismatched parts are output as a message in the message output section 97-2 of the determination result screen 90-2. In addition, a window 98 is also output on the determination result screen 90-2. The window 98 prevents the worker P from proceeding to the next step unless the worker P inputs the release key, and prompts the worker P to confirm. In addition, if the weights do not match, in order to distinguish this from a case where the character strings do not match, the control unit 10 (confirmation result output unit 23) may, for example, alert the worker P by emitting a warning sound from the speaker 6.

[0047] Although not shown, even if the character strings acquired by simply performing OCR processing on the collation region match and the weights match, if the character strings acquired by performing OCR processing after preprocessing do not match or if a mismatch is determined based on image similarity, the control unit 10 outputs a determination result screen in which a "△" mark is output to the result output unit 96 of the determination result screen 90. Then, for example, a window 98 like the determination result screen 90-2 in Fig. 10 described above is output to prompt the operator P to check visually or the like.

[0048] Thereafter, in S17 of Fig. 4, the control unit 10 determines whether or not to end the process. For example, when confirmation of the enclosed documents is completed, the control unit 10 determines that the process should be ended when the worker P performs an operation for ending the process (not shown). If the process should be ended (S17: YES), the control unit 10 ends this process. On the other hand, if the process should not be ended (S17: NO), the control unit 10 moves the process to S11 and remains in this process until the worker P operates the switch 8.

[0049] As described above, the enclosure confirmation system 100 of this embodiment has the following advantages. (1) The enclosure confirmation device 1 extracts a matching area containing key information from a document image, performs OCR processing on the matching area to obtain a character string, performs preprocessing on the matching area to remove noise, and then performs OCR processing on the matching area to obtain a character string, and compares the obtained character strings. The enclosure confirmation device 1 also compares images of the matching areas between documents to calculate image similarity. The enclosure confirmation device 1 then determines whether the key information matches based on the matching result obtained through OCR processing and the calculated image similarity. Therefore, by using image similarity, it is possible to supplement the accuracy of OCR and prevent documents from being mis-enclosed.

[0050] (2) The enclosure check device 1 compares the images of each character in the matching area between the documents and calculates the image similarity for each character. Therefore, by using the image similarity for each character, it is possible to calculate the similarity in smaller units and use it for matching.

[0051] (3) The enclosure check device 1 stores a matching area for each document, and by recognizing the document type from the document image, the enclosure check device 1 can obtain a matching area corresponding to the recognized document type from the document image. Then, the enclosure check device 1 compares and verifies the character strings corresponding to the matching area. Therefore, the enclosure check device 1 can obtain the verification area by determining the type of document.

[0052] (4) The enclosure verification device 1 performs preprocessing on the document image and then performs OCR processing to obtain a character string, and compares the obtained character string with a character string obtained by performing OCR processing before performing preprocessing. Therefore, the enclosure checking device 1 can not only simply use the OCR processed data to determine whether the key information matches, but can also use the data that has been pre-processed and then OCR processed to make the determination, and by adding measures to compensate for the accuracy of the OCR, it is possible to prevent documents from being erroneously enclosed.

[0053] (5) The enclosure verification device 1 compares the acquired character strings by performing OCR processing on the comparison area to compare the character strings obtained, and by performing preprocessing to remove noise and then performing OCR processing to compare the character strings obtained, and determines whether the key information matches based on each comparison result. Therefore, matching between character strings obtained by performing OCR processing after preprocessing to remove noise can be used to compensate for matching errors caused by matching between character strings obtained by performing OCR processing on the matching area. In particular, the risk is greatest when the character strings obtained by OCR processing match even though the key information does not match.Even in such cases, the risk of a match being determined even though the key information actually does not match can be reduced by using the results of matching character strings obtained by OCR processing after performing preprocessing to remove noise.

[0054] (6) The enclosure checker 1 uses multiple preprocessing methods, allowing it to include preprocessing suited to the characteristics of the image in the matching area. This improves the accuracy of the matching process, which would otherwise determine that the key information matches when it actually does not.

[0055] (7) The enclosure check device 1 obtains key information from the obtained character string, and outputs the determination result, which further includes the key information and an image of the collation area, to the monitor 4. Therefore, information can be provided to the worker P in a display format that allows the worker P to easily visually check whether the key information matches.

[0056] (8) The apparatus is provided with a table 40 on which a plurality of documents can be placed side by side, and a camera 5 capable of photographing the table 40, and images of the plurality of documents placed on the table 40 are extracted from the image photographed by the camera 5. Therefore, by simply placing documents to be enclosed in the same envelope side by side on the table 40, the key information can be verified.

[0057] (9) Weight sensors 71 to 74 are provided in the document placement areas of the placement table 40, and it is confirmed whether the weight of each document detected by the weight sensors 71 to 74 matches the weight of each document stored in the document memory unit 32, which stores the weight of each of multiple documents.If it is confirmed that the weights do not match, an alert is issued. Therefore, documents to be enclosed in the same envelope can be confirmed by weight, and it can be determined whether documents to be enclosed in the next envelope have been mixed in, for example.

[0058] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments. Furthermore, the effects described in the embodiments are merely a list of the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments. Note that the above-described embodiments and the modified embodiments described below can be used in appropriate combinations, but detailed description thereof will be omitted.

[0059] (Variations) (1) In the present embodiment, the enclosure confirmation device compares character strings obtained by performing OCR processing on the matching area, compares character strings obtained by performing OCR processing on the matching area after preprocessing, and makes a determination based on the image similarity for each character calculated by comparing images of each character in the matching area across multiple documents. However, this is not limited to this. For example, it may only compare character strings obtained by performing OCR processing on the matching area after preprocessing, and omit the determination based on the image similarity for each character calculated by comparing images of each character in the matching area across multiple documents. Alternatively, it may only compare character strings obtained by performing OCR processing on the matching area after preprocessing, and omit the determination based on the image similarity for each character calculated by comparing images of each character in the matching area across multiple documents. In either case, misinserted documents can be prevented by not only comparing character strings obtained by performing OCR processing on the matching area but also using other processes that supplement the accuracy of OCR.

[0060] (2) In this embodiment, all documents contain key information, but the present invention is not limited to this. It can also be used when documents containing the same content, such as pamphlets sent to all customers, are enclosed in the same envelope. In this case, OCR verification can be omitted for the portions of the document that do not contain the key information, and detection by the weight sensor can be performed. In this way, it is possible to check whether the enclosed items are correct, including the order in which they are enclosed, even when documents such as pamphlets are enclosed.

[0061] (3) In the present embodiment, the verification area is registered in advance, but the present invention is not limited to this. For example, the enclosure confirmation device may identify a verification area from a character string obtained by performing OCR on a document image, and then compare the character string corresponding to the identified verification area to verify the verification. In this way, the enclosure checker does not need to register the verification area in advance, and can obtain the verification area from the character string included in the document.

[0062] (4) In this embodiment, four different types of pre-processing have been described, but the present invention is not limited to this. It is not necessary to use all four types of pre-processing, and it is sufficient to use one or more types of pre-processing. (5) In this embodiment, no particular explanation is given about the system configuration of the enclosed item verification device, but it may be a standalone device configured to perform matching processing independently within the terminal, or it may be a device that has a control unit and memory unit in a server and processes the processing on the server. [Explanation of symbols]

[0063] 1. Enclosure check device 4 monitors 5. Camera 6 speakers 8 Switch 10 Control Unit 11 Start reception desk 12 Image acquisition unit 13 Image extraction section 14 Matching area identification unit 15 Character recognition section 16 Pretreatment section 17 Preprocessed character recognition section 18 Character matching section 19 Output string acquisition section 20 Image similarity calculation unit 21 Character judgment section 22 Weight confirmation section 23 Confirmation result output section 30 Storage section 32 Document storage section 33 Enclosure order memory section 40 Mounting table 41, 42, 43, 44 Placement area 51a, 52a, 53a, 54a Documents 55 envelope 60 screens 61, 62, 63, 64 Document area 71, 72, 73, 74 Weight sensors 90,90-2 Judgment result screen 100 Enclosure Verification System P worker

Claims

1. A character matching device for matching a plurality of types of documents having the same identification character information, a recognition processing means for performing character recognition processing on the image of the document; a pre-processing means for pre-processing the image of the document; a post-preprocessing recognition processing means for performing character recognition processing on the image preprocessed by the preprocessing means; a character matching means for matching each character string corresponding to the plurality of documents obtained by the recognition processing means with each character string corresponding to the plurality of documents obtained by the preprocessing recognition processing means; an image similarity calculation means for comparing images of regions to be matched between documents and calculating image similarity; a determination means for determining whether the identification character information matches based on the comparison result by the character comparison means and the image similarity calculated by the image similarity calculation means; a determination result output means for outputting the determination result by the determination means to a display unit; A character matching device comprising:

2. 2. The character matching device according to claim 1, The image similarity calculation means is a character matching device that compares images of characters in the area to be matched between documents to calculate image similarity.

3. 3. The character matching device according to claim 1, an area specifying means for specifying an area to be matched from the character string obtained by the recognition processing means; The character matching means performs matching by comparing character strings corresponding to the areas to be matched identified by the area identifying means.

4. 3. The character matching device according to claim 1, The area to be matched is stored for each document, a type recognition means for analyzing an image of the document and recognizing the type of the document; The character matching means compares and matches each character string corresponding to the area to be matched for each document based on the type of document recognized by the type recognition means.

5. A character matching device according to any one of claims 1 to 4, The preprocessing means performs a plurality of types of preprocessing on the image of the document.

6. 6. The character matching device according to claim 1, an output character string acquisition means for acquiring an output character string; The determination result output means further outputs the output character string acquired by the output character string acquisition means and an image of each area to be matched.

7. A program for causing a computer to function as the character matching device according to any one of claims 1 to 6.

8. An enclosure confirmation system including the character matching device according to any one of claims 1 to 6, a placement table on which a plurality of the document enclosures can be placed side by side; an imaging unit capable of imaging the mounting table; an image acquisition unit that acquires, via the photographing unit, photographed images including images of the plurality of documents placed on the document table; an image cutting means for cutting out an image of each document from the photographed image acquired by the image acquisition means; An enclosure confirmation system comprising:

9. In the enclosed item confirmation system according to claim 8, a weight detection unit provided in a placement area of ​​each enclosure on the placement table; a weight storage unit for storing the weight of each type of enclosed item; a weight confirmation means for confirming whether the weight of each enclosure detected by the weight detection unit matches the weight of each enclosure stored in the weight memory unit; a notification means for notifying when the weight confirmation means confirms that the weights do not match; An enclosure confirmation system comprising:

Images