Image processing device, image processing method, and program

Abstract

To provide an image processing apparatus capable of correcting an orientation of a scanned image, even if the orientation of a document during scanning is incorrect.SOLUTION: An image processing apparatus includes: a document type determination unit configured to determine a document type of image data obtained by optical reading; a selection unit configured to select, based on a determination result from the document type determination unit, a determination algorithm to be applied, from a plurality of determination algorithms for determining an orientation of an image; and an orientation determination unit configured to determine an orientation of an image in the image data by using the determination algorithm selected by the selection unit.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] For example, Patent Document 1 discloses an image processing device having an input image acquisition unit 161 that acquires an input image generated by reading a document, a character string information calculation unit 168 that calculates character string density or character string variance in the input image, and a document type discrimination unit 169 that discriminates the type of document based on the character string density or character string variance.

[0003] Patent document 2 also discloses an image processing device 200 having a memory unit 210 that stores the position of a specified area in an image according to a specified format, an image acquisition unit 221 that acquires an input image including an area to be recognized as a character, a corrected image generation unit 223 that cuts out an area corresponding to the area to be recognized as a character from the input image or an image generated from the input image, and generates a corrected image in which the area is cut out and placed at the position of the specified area in the image according to the specified format, a character detection unit 224 that detects characters from the corrected image, and an output unit 203 that outputs information related to the detected characters. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6161484 [Patent Document 2] Patent Publication No. 2020-086847 Summary of the Invention [Problem to be solved by the invention]

[0005] To provide an image processing device capable of correcting the orientation of a scanned image even if the orientation of a document is mistaken when scanning. [Means for solving the problem]

[0006] The image processing device of the present invention has a document type determination unit that determines the document type from optically read image data, a selection unit that selects a determination algorithm to be applied from a plurality of determination algorithms that determine the orientation of an image based on the determination result by the document type determination unit, and an orientation determination unit that determines the orientation of the image in the image data using the determination algorithm selected by the selection unit.

[0007] Preferably, the image processing device further includes an area extraction unit that extracts an image area corresponding to a document from the image data, and the document type determination unit determines the document type based on the size or shape of the image area extracted by the area extraction unit.

[0008] Preferably, the device further includes an MRZ search unit that searches the image data for a machine-readable area defined by the International Civil Aviation Organization, and the document type determination unit determines the document type based on the search results by the MRZ search unit.

[0009] Preferably, the document type determination unit determines whether the document of the image data is a travel document, and when the document type determination unit determines that the document of the image data is a travel document, the selection unit selects a determination algorithm for searching for a machine-readable area while rotating the image data by 90 degrees.

[0010] Preferably, when the document type determination unit determines that the document of the image data is not a travel document, the selection unit rotates the image data by 90 degrees, 180 degrees, or 270 degrees and selects a determination algorithm for performing optical character recognition processing.

[0011] Preferably, the image processing apparatus further includes a binarization unit which performs a binarization process on image data of the image area extracted by the area extraction unit to generate a binary image, the document type determination unit determines the document type using the binary image generated by the binarization unit, the orientation determination unit determines the image orientation in the image data using the binary image generated by the binarization unit, and further includes an original image rotation unit which rotates the image data in accordance with a result of the determination by the orientation determination unit.

[0012] Preferably, the document further comprises a front / back swapping section which, when a plurality of image data read by double-sided scanning are input, swaps the front and back images based on the result of the determination by the orientation determination section.

[0013] Preferably, the image forming apparatus further includes an angle selection unit which selects a rotation angle in a priority order corresponding to the document type determined by the document type determination unit, and the orientation determination unit determines the orientation of the image based on the image data rotated by the rotation angle selected by the angle selection unit.

[0014] The image processing method of the present invention includes a document type determination step for determining a document type for optically read image data, a selection step for selecting a determination algorithm to be applied from a plurality of determination algorithms for determining the orientation of the image based on the determination result in the document type determination step, and an orientation determination step for determining the orientation of the image in the image data using the determination algorithm selected in the selection step.

[0015] The program of the present invention causes a computer to execute an original type determination step of determining an original type for optically read image data, a selection step of selecting an application of a determination algorithm from among a plurality of determination algorithms for determining the orientation of the image based on the determination result in the original type determination step, and an orientation determination step of determining the orientation of the image in the image data using the determination algorithm selected in the selection step. Effect of the Invention

[0016] To provide an image processing device capable of correcting the orientation of a scanned image even if the orientation of a document is mistaken when scanning. [Brief description of the drawings]

[0017] [Figure 1] 1 is a diagram illustrating an example of the overall configuration of an image assessment system 1. FIG. [Diagram 2] FIG. 13 is a diagram illustrating an example of failure in grouping character strings in MRZ. [Diagram 3] 11A and 11B are diagrams illustrating examples of erroneous determination factors in OCR processing in MRZ. [Figure 4] 2 is a diagram illustrating an example of a hardware configuration of an image processing device 2. FIG. [Diagram 5] 2 is a diagram illustrating an example of a functional configuration of an image processing device 2. FIG. [Figure 6] 10 is a flowchart illustrating image processing (S10) by the image processing device 2. [Figure 7] 13A and 13B are diagrams illustrating examples of determination algorithms and priorities of rotation angles corresponding to document types. [Figure 8] FIG. 13 is a diagram illustrating a front / back switching process. [Figure 9] 13 is a flowchart illustrating the MRZ search process (S30) in more detail. [Figure 10] FIG. 13 is a diagram illustrating a projection histogram. [Figure 11] 13 is a diagram illustrating a modified example of the document type determination unit 320. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] [background] The background of the present invention will be described. When hotels and other establishments are accepting travelers from overseas, document scanners can be placed at the reception desk to scan documents such as passports and ID cards to obtain customer information and help with management and security at the hotel, in order to verify the identity of the travelers. In other words, there is a scenario in which a scanner can be used to perform OCR recognition on a scanned image of a certificate and obtain personal information such as the customer's name. The certificates used in this type of business are known as Travel Documents, and generally have a character string area (machine-readable area) dedicated to OCR, called MRZ (Machine Readable Zone). Travel documents have internationally unified regulations set by ICAO (International Civil Aviation Organization) and other organizations regarding the dimensions of the document, the position of the MRZ area, the number of columns and characters in the MRZ, etc., and are documents that generally conform to the regulations of each country. Examples of certificates that travelers carry and are commonly used at counters include TD1 (ID card) and TD3 (passport) (hereinafter referred to as TD1 and TD3).

[0019] However, while the size of the manuscript and the dimensions of the MRZ position, etc., for TD1 and TD3 must conform to prescribed standards, the background design, color, and the placement and color of certification text information other than MRZ are left to the discretion of each country. In addition, identity authentication at the counter is essentially an ad-hoc task, and when an operator uses a scanner or other device, there may be cases where they make mistakes with the scanning direction (the direction in which the document is placed) or orientation (front and back). Furthermore, in the case of an unmanned reception desk, if travelers themselves operate the scanner, it will be necessary to take into consideration the above-mentioned precautions against scanning errors. In addition, considering that the scanner is also used for other tasks at reception, it would be time-consuming to have to switch the scan mode used each time a traveler is checked in. To eliminate the need to switch between these types of documents, it would be effective to automatically recognize that it is a travel document.

[0020] FIG. 1 is a diagram illustrating an example of the overall configuration of an image processing system 1. As shown in FIG. 1, the image processing system 1 includes an image processing device 2 and a scanner device 4, which are connected to each other via a cable 7. In this example, a specific example of connection using a cable 7 such as a USB cable will be described, but the present invention is not limited to this, and for example, a wireless connection may be used. In addition, the function of the image processing device 2 may be built into the housing of the scanner device 4. The image processing device 2 is a computer terminal, and processes image data read by the scanner device 4. Specifically, the image processing device 2 determines the orientation of the image data optically read by the scanner device 4, and rotates the image data so that the image is upright. The scanner device 4 is an optical image reading device.

[0021] 2 and 3 are diagrams for explaining problems in automatic recognition processing of travel documents. In orientation determination methods based on OCR processing, the key is to extract character strings, but in the MRZ regions of TD1 and TD3, the following types of determination errors can occur, and when it comes to TD1 and TD3 alone, the accuracy of orientation determination is not high. That is, there are cases where the joining group of character strings is mistaken for vertical or horizontal, as shown in the example of Figure 2. Character strings are grouped based on the characteristic that the characters are arranged in an orderly manner, but in MRZ regions, as shown in Figure 2, the characters are arranged in an orderly manner in both the vertical and horizontal directions, making it easy to make mistakes. The orderliness of character strings in MRZ regions is due to the fact that the number of characters is fixed and that proportional fonts are not used (character spacing is constant).

[0022] In addition, as shown in Figure 3(A), the "<" character is used frequently in the MRZ area. This character is used as a supplementary invalid character to fill up the number of characters in the MRZ area. "<" is often misjudged as "VVV" or "^^^" in OCR processing, which can lead to incorrect orientation detection. Furthermore, since TD1 and TD3 are personal certificates, various design patterns are used in the background of the characters, as shown in Fig. 3(B), making it difficult to extract the character strings in the first place. In addition, if the background pattern is removed and OCR processing is performed, the processing time becomes long. Therefore, the image processing device 2 of the present embodiment determines the document type of the scanned image data, and determines the image orientation in the image data using a determination algorithm corresponding to the document type. The determination algorithm is, for example, an algorithm for searching the MRZ area and an algorithm for executing the OCR process.

[0023] FIG. 4 is a diagram illustrating an example of a hardware configuration of the image processing device 2. As shown in FIG. As illustrated in FIG. 4, the image processing device 2 includes a CPU 200, a memory 202, a HDD 204, a network interface 206 (network IF 206), a display device 208, and an input device 210, which are connected to each other via a bus 212. The CPU 200 is, for example, a central processing unit. The memory 202 is, for example, a volatile memory, and functions as a main storage device. The HDD 204 is, for example, a hard disk drive device, and serves as a non-volatile storage device for storing computer programs (for example, the image processing program 3 in FIG. 5) and other data files. The network IF 206 is an interface for wired or wireless communication, and realizes communication with the scanner device 4, for example. The display device 208 is, for example, a liquid crystal display. The input device 210 is, for example, a keyboard and a mouse.

[0024] FIG. 5 is a diagram illustrating an example of the functional configuration of the image processing device 2. As shown in FIG. 5, an image processing program 3 is installed in the image processing device 2. The image processing program 3 has a region extraction unit 300 , a binarization unit 310 , a document type determination unit 320 , an algorithm selection unit 330 , an angle selection unit 340 , a direction determination unit 350 , an original image rotation unit 360 , and a front / back swap unit 370 . Note that a part or the whole of the image processing program 3 may be realized by hardware such as an ASIC, or may be realized by borrowing some of the functions of an OS (Operating System).

[0025] In the image processing program 3, the area extraction unit 300 extracts an image area corresponding to an original from image data scanned by the scanner device 4. For example, the outside of the original area has a backing (monochrome) that serves as a certain background color, and the area extraction unit 300 extracts only the original portion from the image including the backing background. The area extraction unit 300 may also extract an area corresponding to the content (content area) from the scanned image as the original part. This is because the image area with margins (blank space) added around the content area is considered to be the original area. In this case, even if a copy of a travel document is scanned by the scanner device 4, an image can be extracted based on the shape and size of the travel document, not the shape and size of the copy paper.

[0026] The binarization unit 310 generates a binary image by performing binarization processing on the image data of the image region extracted by the region extraction unit 300. The threshold value of the binarization processing is, for example, a density value at which the background pattern can be removed and the characters can remain. The binarization unit 310 may also perform binarization and resolution reduction processing on the image data of the image region extracted by the region extraction unit 300 to generate a binary image with a lower resolution than the original image. This can reduce the processing load of the document type discrimination process and image orientation determination process.

[0027] The document type determination unit 320 determines the document type of the image data by using the binary image generated by the binarization unit 310. The document type to be determined is, for example, whether or not it is a travel document. The document type determination unit 320 of this example determines whether the document of the scanned image data is the card size of TD1, the passport spread size of TD3, or other size based on the size and shape of the binary image generated by the binarization unit 310.

[0028] The algorithm selection unit 330 selects an applied determination algorithm from among a plurality of determination algorithms for determining the orientation of an image, based on the determination result by the document type determination unit 320. In this example, as illustrated in Fig. 7, the algorithm selection unit 330 selects an algorithm ("Algo 1") for searching the MRZ region while rotating the image data when the document type determination unit 320 determines that the document is TD1 or TD3, and selects an algorithm ("Algo 2") for performing OCR processing while rotating the image data when the document type determination unit 320 determines that the document is other than TD1 or TD3.

[0029] The angle selection unit 340 selects the rotation angle in a priority order corresponding to the document type determined by the document type determination unit 320. As illustrated in FIG. 7, the angle selection unit 340 in this example selects the rotation angle in a priority order corresponding to the document type determined by the document type determination unit 320 and the shape (aspect ratio) of the area extracted by the area extraction unit 300. As illustrated in FIG. 7, when the document is a travel document (TD1 or TD3), the orientation determination unit 350 can complete the determination process with two rotation angles. Note that when the document is other than a travel document, the orientation determination unit 350 rotates the image data in a default order (0 degrees, 90 degrees, 180 degrees, 270 degrees).

[0030] The orientation determination unit 350 determines the orientation of the image in the image data using the determination algorithm selected by the algorithm selection unit 330. In this example, the orientation determination unit 350 performs a search process or an OCR process for the MRZ region on the image data rotated at the rotation angle selected by the angle selection unit 340 according to the algorithm selected by the algorithm selection unit 330, to determine the orientation of the image in the image data.

[0031] The original image rotation unit 360 rotates the original image data acquired from the scanner device 4 according to the result of the determination by the orientation determination unit 350 . When multiple image data read by double-sided scanning are input, the front / back image swapping unit 370 swaps the front and back images based on the determination result by the orientation determination unit 350. For example, when the document of the image data read by double-sided scanning is determined to be TD1 by the document type determination unit, the front / back image swapping unit 370 swaps the front and back images so that the image data in which the MRZ area is found by the orientation determination unit 350 becomes the back side, as illustrated in FIG.

[0032] FIG. 6 is a flowchart illustrating the image processing (S10) performed by the image processing device 2. As illustrated in FIG. 6, in step 100 (S100), the image processing device 2 (image processing program 3 in FIG. 5) acquires image data scanned by the scanner device 4. In step 105 (S105), the binarization unit 310 of the image processing program 3 performs binarization processing on the image data acquired from the scanner device 4 to generate a binary image. In step 110 (S110), the region extraction section 300 determines that a region in the binary image generated by the binarization section 310 in which a predetermined amount or more of black pixels exist is a content region, and extracts the content region.

[0033] In step 115 (S115), the document type determination section 320 determines, based on the shape and size of the content area extracted by the area extraction section 300, whether the document of the image data is TD1, TD3, or something else. In step 120 (S120), if the document type determination unit 320 determines that the document of the image data is a travel document (TD1 or TD3), the image processing program 3 proceeds to processing of S125, and if the document is determined to be other than a travel document, the image processing program 3 proceeds to processing of S135.

[0034] In step 125 (S125), the angle selection unit 340 determines the priority of the rotation angles based on the document type (TD1 or TD3) determined by the document type determination unit 320 and the aspect ratio of the content area extracted by the area extraction unit 300, as illustrated in FIG. In step 30 (S30), the algorithm selection unit 330 selects an algorithm (Algorithm 1) for searching for an MRZ region while rotating the image data to determine the orientation of the image, and the orientation determination unit 350 determines the orientation of the image in the image data according to the selected algorithm. Note that instead of rotating the image data, the coordinate information of the MRZ may be rotated and searched. Also, in S30, if the MRZ region is not found in any of the rotation directions, the flow may proceed to S135.

[0035] In step 135 (S135), the algorithm selection unit 330 selects an algorithm (Algo 2) that performs OCR processing while rotating the image data to determine the orientation of the image, and the orientation determination unit 350 performs OCR processing on the image data rotated 0 degrees, the image data rotated 90 degrees, the image data rotated 180 degrees, and the image data rotated 270 degrees in accordance with the selected algorithm, and determines the orientation of the image based on the accuracy of the character recognition processing.

[0036] In step 140 (S140), the original image rotation unit 360 rotates the original image data acquired from the scanner device 4 based on the result of the determination by the orientation determination unit 350, to generate image data in an upright state. In step 145 (S145), the front / back replacement unit 370 acquires double-sided scanned image data from the scanner device 4, and if the document type determination unit 320 determines that the image data is TD1, the process proceeds to S145; otherwise, the image processing (S10) is terminated. In step 150 (S150), the front / back swapping unit 370 swaps the front / back (order) of the image data scanned on both sides so that the image data in which the MRZ area has been found by the orientation determining unit 350 becomes the image data on the back side.

[0037] FIG. 9 is a flowchart explaining the MRZ search process (S30) in more detail. 9, in step 300 (S300), the orientation determination unit 350 rotates the binary image generated by the binarization unit 310 at the rotation angle of priority 1 determined by the angle selection unit 340. Note that if the rotation angle is 0 degrees, the rotation process is not performed and the image is skipped. In step 305 (S305), the orientation determination unit 350 generates a vertically projected black pixel histogram from the rotated binary image, as exemplified in FIG.

[0038] In step 310 (S310), the orientation determination unit 350 determines whether or not there are pixels of a character string with a predetermined number of lines at a predetermined interval on the bottom side of the image in the generated vertical projection histogram as shown in Fig. 10. If there are, the process proceeds to S315, and if there are no pixels, the process proceeds to S330. The predetermined number of lines is, for example, 3 lines in the case of TD1 and 2 lines in the case of TD3.

[0039] In step 315 (S315), the orientation determination unit 350 generates a histogram of horizontally projected black pixels from the binary image, limited to the lower region of the image (the region having a predefined number of rows of pixels), as illustrated in FIG. 10. In step 320 (S320), the orientation determination unit 350 determines whether or not there are pixels corresponding to a predetermined number of characters in the generated horizontal projection histogram, as shown in Fig. 10. If there are, the process proceeds to S325, and if there are no pixels, the process proceeds to S330. The predetermined number is, for example, 30 characters in the case of TD1 and 44 characters in the case of TD3.

[0040] In step 325 (S325), the orientation determining unit 350 determines that an MRZ region exists, and determines the orientation of the image based on the angle by which the binary image is rotated. In step 330 (S330), if there is a rotation angle with the next priority determined by the angle selection unit 340, the orientation determination unit 350 returns to the process of S300 and executes the processes of S300 to S325 with the rotation angle with the next priority. If there is no rotation angle with the next priority determined by the angle selection unit 340, the orientation determination unit 350 proceeds to the process of S335. In step 335 (S335), the orientation determining unit 350 determines that an MRZ area does not exist.

[0041] As described above, according to the image processing system 1 of the present embodiment, it is possible to determine the document type of image data scanned by the scanner device 4 and to determine the image orientation in the image data using an algorithm corresponding to the document type. In particular, by switching the algorithm depending on whether the document type is a travel document or not, it is possible to avoid erroneous determinations such as those illustrated in Figs. 2 and 3 and improve the determination accuracy.

[0042] (Modification) Next, a modification of the above embodiment will be described. FIG. 11 is a diagram illustrating a modified example of the document type determination unit 320. In FIG. As illustrated in Fig. 11, the document type determination unit 320 of the modified example includes an MRZ search unit 322. The MRZ search unit 322 determines the presence or absence of an MRZ area by the same process as in Fig. 9. The document type determination unit 320 of the modified example determines the document type based on the search result by the MRZ search unit 322.

[0043] Also, in the above embodiment, when the original is a travel document, the orientation determination unit 350 searches the MRZ area to determine the orientation of the image, but this is not limited to this, and for example, the orientation of the image may be determined by analyzing the line structure. In addition, the priority of the rotation angles may be determined in order from the most probable direction, taking into account the tendency of setting documents on the scanner device 4. For example, if the scanner device 4 is marked or labeled with a sticker indicating the method of setting the document, the priority of that direction may be set to 1. [Explanation of symbols]

[0044] 1. Image processing system 2. Image processing device 3. Image processing program 4. Scanner device 300...Area extraction part 310...Binarization section 320...Document type determination section 330…Algorithm selection section 340...Angle selection section 350…Orientation determination section 360…Original image rotation section 370…Front and back interchangeable section

Claims

1. a document type determination unit that determines whether the optically read image data is a document including MRZ as a document type; a selection unit that selects an application of a determination algorithm from among a plurality of determination algorithms for determining an orientation of an image based on a determination result by the document type determination unit; an orientation determination unit that determines an orientation of an image in the image data by using the determination algorithm selected by the selection unit; having The selection unit is an image processing device that, when the document type determination unit determines that the document of the image data is a document including MRZ, selects a determination algorithm for searching for a machine-readable area while rotating the image data by 90 degrees.

2. a region extraction unit for extracting an image region corresponding to a document from the image data; and The document type determination unit determines the document type based on the size or shape of the image area extracted by the area extraction unit. The image processing device according to claim 1 .

3. An MRZ search unit that searches the image data for a machine-readable region defined by the International Civil Aviation Organization. and The document type determination unit determines the document type based on a search result by the MRZ search unit. The image processing device according to claim 1 .

4. The selection unit, when the document type determination unit determines that the document of the image data is not a document including MRZ, rotates the image data by 90 degrees, 180 degrees, or 270 degrees and selects a determination algorithm for performing optical character recognition processing. The image processing device according to any one of claims 1 to 3.

5. a binarization unit that performs binarization processing on the image data of the image region extracted by the region extraction unit to generate a binary image; and The document type determination unit determines a document type by using the binary image generated by the binarization unit; The orientation determination unit determines an orientation of an image in the image data by using the binary image generated by the binarization unit; an original image rotation unit that rotates the image data in accordance with a result of the determination by the orientation determination unit; The image processing device according to claim 2 , further comprising:

6. a front / back swapping section that swaps the front and back images based on a result of the determination by the orientation determining section when a plurality of image data sets read by double-sided scanning are input; 6. The image processing device according to claim 1, further comprising:

7. an angle selection section for selecting a rotation angle in accordance with a priority order corresponding to the document type determined by the document type determination section; and The orientation determination unit determines an orientation of an image based on the image data rotated by the rotation angle selected by the angle selection unit. The image processing device according to any one of claims 1 to 6.

8. a document type determination step of determining whether the optically read image data is a document including MRZ as a document type; a selection step of selecting an application determination algorithm from among a plurality of determination algorithms for determining the orientation of an image based on a determination result in the document type determination step; an orientation determination step of determining an orientation of an image in the image data by using the determination algorithm selected by the selection step; having The selection step is an image processing method in which, when the document type determination step determines that the document of the image data is a document including MRZ, a determination algorithm is selected to search for a machine-readable area while rotating the image data by 90 degrees.

9. a document type determination step of determining whether the optically read image data is a document including MRZ as a document type; a selection step of selecting an application determination algorithm from among a plurality of determination algorithms for determining the orientation of an image based on a determination result in the document type determination step; an orientation determination step of determining an orientation of an image in the image data by using the determination algorithm selected by the selection step; on the computer, The selection step is a program for selecting a determination algorithm for searching for a machine-readable area while rotating the image data by 90 degrees when the document type determination step determines that the document of the image data is a document including MRZ.

Images