Processing program, processing apparatus, and processing method

Abstract

To provide a processing program or the like for accurately performing processing for extracting characters from the image data of a document to which a revenue stamp is stuck.SOLUTION: The character-extraction-processing program 8a causes a computer to function as an image-data-acquiring unit 8b that acquires contract-image-data 4a, a stamp-area-determining unit 8b that performs image processing such as SIFT on the contract-image-data 8b to determine a stamp-area R22 in the contract-image-data 4b corresponding to an area where a revenue stamp is affixed in a contract, and a character-extracting unit 8b that performs processing of extracting characters from an area other than the stamp-area R22 in the contract-image-data. 4c.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a processing program, a processing device, and a processing method for extracting characters from image data. [Background technology]

[0002] Conventionally, there has been known a technique for acquiring image data of a predetermined document, such as a contract, using a scanner or camera and extracting characters from the image data using optical character recognition (OCR: Optical Character Recognition (or Reader)). OCR is a technique for recognizing characters in image data and converting them into text data (in other words, character code strings), that is, a technique for digitizing characters in image data.

[0003] This type of technology is disclosed, for example, in Patent Documents 1 to 3. Patent Document 1 discloses a technology for detecting a seal imprint in a document image acquired by a scanner and performing character recognition on the document image from which the seal imprint has been removed. This technology removes the vermilion portion, which is the color of the seal imprint, from the document image. Patent Document 2 discloses a technology for extracting a specific pattern, such as a seal imprint, from a form image containing multiple overlapping print patterns. This technology extracts the seal imprint portion based on the color distribution of each pixel that makes up the form image. Patent Document 3 discloses a technology for reading grayscale image data obtained by scanning a business document in grayscale, searching for a seal imprint area within the grayscale image data, and removing a polygonal area containing the seal imprint. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-157375 [Patent Document 2] Japanese Patent Application Laid-Open No. 2008-176521 [Patent Document 3] Japanese Patent Application Laid-Open No. 2010-157107 Summary of the Invention [Problem to be solved by the invention]

[0005] Incidentally, revenue stamps are affixed to documents such as contracts and receipts (such as receipts). A "revenue stamp" is a certificate issued by the government to collect taxes, fees, and other payments that constitute national revenue. When OCR is performed on the image data of a document with such a revenue stamp affixed, the characters on the revenue stamp are also extracted. This will be explained in detail using the example shown in Figure 12. Figure 12 is an explanatory diagram of the problems that can arise when OCR is performed on a contract with a revenue stamp affixed.

[0006] The top part of Figure 12 shows a portion of a contract with a revenue stamp affixed. When OCR is performed on the image data of such a contract, as shown in the bottom part of Figure 12, not only the contract title ("Construction Work Contract") and the contract content (e.g., "●● (hereinafter referred to as the "Client") and ●● (hereinafter referred to as the "Contractor")," but also the characters "Revenue Stamp," "Japanese Government," and "10,000 Yen" from the revenue stamp are extracted. As a result, in systems or software that automatically set the topmost character string (corresponding to the characters at the top of a document) extracted by OCR as the title of the document's text data, the characters from the revenue stamp are included in the title of the text data, since revenue stamps tend to be affixed to the top of documents. In other words, in the example shown in Figure 12, "Revenue Stamp Japanese Government 10,000 Yen Construction Work Contract" is set as the title of the text data.

[0007] On the other hand, while the above-mentioned Patent Documents 1 to 3 describe processing of the seal imprint in the image data (detection of the seal imprint, removal of the seal imprint, etc.), they do not describe or suggest anything about processing the revenue stamp in the image data. The seal imprint in the image data can typically be processed based on the color (vermilion) specific to the seal imprint, but because revenue stamps include various colors, even if such processing of the seal imprint is applied, it is not possible to process the revenue stamp accurately.

[0008] The present invention has been made to solve the above-mentioned problems, and aims to provide a processing program, processing device, and processing method that can accurately extract characters from image data, even for documents with revenue stamps affixed. [Means for solving the problem]

[0009] In order to achieve the above object, the present invention is a processing program executed by a computer to perform processing to extract characters from image data, and is characterized in that it causes the computer to function as an image data acquisition unit that acquires image data of a specified document, a stamp area determination unit that performs specified image processing on the image data to determine a stamp area in the image data that corresponds to an area on the document where a revenue stamp is affixed, and a character extraction unit that performs processing to extract characters from areas in the image data other than the stamp area.

[0010] According to the present invention configured in this manner, the stamp area determination unit determines a stamp area in the image data corresponding to an area where a revenue stamp is affixed in a predetermined document (e.g., a contract), and the character extraction unit performs a process of extracting characters from areas of the image data other than the stamp area. This appropriately prevents characters in the revenue stamp (such as "Revenue Stamp" or "Japanese Government") from being extracted by OCR or the like for documents with revenue stamps affixed. Therefore, according to the present invention, even for documents with revenue stamps affixed, the process of extracting characters from the image data can be performed accurately.

[0011] In the present invention, the print area determination section preferably determines the print area by performing image processing only on image data corresponding to a specific partial area of ​​the document in the image data of the document. According to the present invention configured as described above, image processing is not performed on the entire image data of a document, but on a limited area of ​​the image data, thereby ensuring the accuracy of image processing and reducing the processing time.

[0012] In the present invention, preferably, the document is made up of one or more pages, and the specific partial area is the upper area of ​​the first page of the document. According to the present invention configured as described above, since revenue stamps tend to be affixed in the upper area of ​​the first page of a document, image processing is performed only on image data corresponding to such an area. This allows the stamp area to be accurately determined from the image data even if the area to be image processed in the image data is limited. In other words, the stamp area can be accurately determined while ensuring the accuracy of image processing and reducing processing time.

[0013] In the present invention, the print area determination section preferably determines the print area by performing image processing only on color portions of the image data. According to the present invention configured in this manner, the stamp area determination unit performs image processing only on color portions of the image data, and does not perform image processing on black portions (nor white portions). In such a case, image processing is not performed on portions of the image data that contain black text, while image processing is performed on portions of the image data that contain revenue stamps, since revenue stamps are color. This allows the portion of the image data to be subjected to image processing to be accurately limited, i.e., the area to be subjected to image processing can be limited to the area where the revenue stamp is affixed. As a result, the stamp area can be accurately determined while ensuring image processing accuracy and reducing processing time.

[0014] In the present invention, preferably, the stamp area determination unit determines the stamp area by performing image processing on the image data, which includes a process of identifying a temporary area within the image data that corresponds to a revenue stamp, and a process of enlarging the temporary area by a magnification corresponding to the size that corresponds to the actual revenue stamp. According to the present invention configured in this manner, even if the initially identified provisional area does not cover the entire revenue stamp, by enlarging this provisional area according to the size corresponding to the actual revenue stamp, it is possible to accurately determine a stamp area that covers the entire revenue stamp.

[0015] In the present invention, the character extraction section preferably extracts characters from an area other than the stamp area in the image data by masking the stamp area in the image data. According to the present invention configured as described above, it is possible to accurately exclude the stamp area from the image data and extract the characters.

[0016] In the present invention, preferably, the stamp area determination unit determines the stamp area by performing image processing on the image data, which involves detecting feature points in the image data and matching the detected feature points with feature points of an image corresponding to an actual revenue stamp. According to the present invention configured as described above, it is possible to determine the print area from the image data with high accuracy.

[0017] In a preferred example of the present invention, the document is a contract. Since revenue stamps tend to be affixed to contracts, the present invention makes it possible to accurately perform processing to extract characters from the image data of contracts to which revenue stamps have been affixed.

[0018] From another perspective, in order to achieve the above-mentioned object, the present invention is a processing device that performs processing to extract characters from image data, and is characterized by having an image data acquisition unit that acquires image data of a specified document, a stamp area determination unit that performs specified image processing on the image data to determine a stamp area in the image data that corresponds to an area on the document where a revenue stamp is affixed, and a character extraction unit that performs processing to extract characters from areas in the image data other than the stamp area. The present invention configured in this manner also makes it possible to accurately extract characters from image data of documents to which revenue stamps are affixed.

[0019] From another perspective, in order to achieve the above-mentioned object, the present invention is a processing method executed by a computer to perform processing to extract characters from image data, characterized in that it comprises an image data acquisition step of acquiring image data of a specified document, a stamp area determination step of performing specified image processing on the image data to determine a stamp area in the image data corresponding to an area on the document where a revenue stamp is affixed, and a character extraction step of performing processing to extract characters from an area in the image data other than the stamp area. The present invention configured in this manner also makes it possible to accurately extract characters from image data of documents to which revenue stamps are affixed. [Effects of the Invention]

[0020] According to the processing program, processing device, and processing method of the present invention, even if a document has a revenue stamp affixed, it is possible to accurately extract characters from the image data. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a schematic diagram illustrating an example of an overall system in which a processing program according to an embodiment of the present invention is executed. [Figure 2] 1 is a block diagram showing an example of the configuration of a server device that executes a processing program according to an embodiment of the present invention; [Figure 3] FIG. 2 is a block diagram showing functional units realized in a server device by a processing program according to an embodiment of the present invention. [Figure 4] 3 is a flowchart showing an outline of the overall process executed by the server device in the embodiment of the present invention. [Figure 5] 10 is a flowchart illustrating a stamp removal process according to an embodiment of the present invention. [Figure 6] 5A and 5B are explanatory diagrams of sample stamp image data used in the embodiment of the present invention. [Figure 7] FIG. 10 is an explanatory diagram of a specific partial area of ​​a contract document, in which feature points are detected from the contract document image in an embodiment of the present invention. [Figure 8] FIG. 10 is an explanatory diagram of a matching process according to an embodiment of the present invention. [Figure 9] 10A and 10B are explanatory diagrams of a method for determining a print area by enlarging a provisional print area in an embodiment of the present invention. [Figure 10] 10 shows an example of a contract image after being subjected to mask processing in an embodiment of the present invention. [Figure 11] 10 shows an example of characters extracted by OCR from a contract to which a revenue stamp has been affixed after the stamp removal process and the seal impression removal process according to an embodiment of the present invention. [Figure 12] This is an explanatory diagram of problems that can occur when performing OCR on a contract to which revenue stamps are affixed. DETAILED DESCRIPTION OF THE INVENTION

[0022] Hereinafter, a processing program, a processing device, and a processing method according to an embodiment of the present invention will be described with reference to the accompanying drawings.

[0023] [System Configuration] First, the system configuration in which the processing program according to this embodiment is executed will be described with reference to Figures 1 to 3. Figure 1 is a schematic diagram showing an example of an overall system in which the processing program according to this embodiment is executed. Figure 2 is a block diagram showing an example of the configuration of a server device that executes the processing program according to this embodiment. Figure 3 is a block diagram showing functional units realized in the server device by the processing program according to this embodiment.

[0024] 1, the processing program according to this embodiment operates in a system in which a server device 1, which is a computer, and a terminal device 2 are communicably connected to each other via a network 3. Here, the terminal device 2 is operated by a user A to convert a contract, which is a predetermined document, into text data for review.

[0025] The server device 1 is a server-type information processing device that operates in response to requests from the terminal device 2, and is equipped with an electronic circuit made up of electronic components such as a CPU (Central Processing Unit) having functions for processing information within the main body, a flash memory, etc. Note that the server device 1 does not necessarily have to be made up of a single information processing device, but may be one in which multiple information processing devices operate in cooperation with each other, or one that operates using any cloud service.

[0026] Terminal device 2 is an information processing device such as a PC (Personal Computer) or tablet terminal device, and includes electronic components such as a CPU with functions for processing information and a flash memory within its main body. Terminal device 2 is also connected to a scanner (image scanner) 20 for generating image data from a contract as a paper document. Scanner 20 optically reads (scans) the contract, generates image data of the contract, and outputs this image data to terminal device 2. Scanner 20 includes not only devices with only a scanning function, but also devices with multiple functions in addition to the scanning function (typically, multifunction devices).

[0027] In the following, the image data of the contract will be referred to as "contract image data," and the image of the contract will be referred to as "contract image." This contract image data is not limited to image data generated by the scanner 20, and image data generated by photographing with a camera may also be used.

[0028] The network 3 is a communication network capable of high-speed communication, and may be, for example, a wired or wireless communication network such as the Internet, an intranet, or a LAN (Local Area Network) that has been provided with security measures.

[0029] In the above system, server device 1, on which the processing program according to this embodiment is installed, executes a process of extracting characters from contract image data, i.e., a process of generating text data from the contract image data, and a process of reviewing the contract from this text data, based on the operation of user A. User A reads the contract using scanner 20, uploads the contract image data generated by scanner 20 to server device 1 via terminal device 2, and then reviews the contract by operating terminal device 2.

[0030] Next, a specific configuration of the server device 1 will be described. As shown in Fig. 2, the server device 1 can include a central processing unit 4, which is a processor, a main memory device 5, an input / output interface device 6, an input device 7, an auxiliary memory device 8, and an output device 9. These devices are connected to one another via a data bus and / or a control bus. The auxiliary memory device 8 is composed of a storage medium such as a hard disk drive or flash memory, and is configured to store information. A character extraction processing program 8a corresponding to the processing program of this embodiment is stored in the auxiliary memory device 8 (see Fig. 3), and is loaded into the main memory device 5 upon execution, whereby processing to extract characters from contract image data is performed.

[0031] In such a hardware configuration, the central processing unit 4 can sequentially load instructions and data (computer programs) constituting a specific application stored in the auxiliary storage device 8 into the main storage device 5 and perform operations on the loaded instructions and data. This allows the central processing unit 4 to control the output device 9 via the input / output interface device 6, and also to send and receive various information (data) to and from other devices (such as the terminal device 2) via the input / output interface device 6 and the network 3.

[0032] As a result, the server device 1 can function as a web server by executing the specific application installed thereon. The server device 1 can then execute various operations (described below) related to the extraction of characters from contract image data via a web page transmitted to the terminal device 2 (and furthermore, a computer program embedded in the web page and executed by the terminal device 2). Alternatively, the server device 1 can function as an application server by executing the specific application installed thereon. This allows the server device 1 to execute similar operations.

[0033] Next, referring to Fig. 3, the functional units realized on the server device 1 by executing the character extraction processing program 8a according to this embodiment will be described. By executing the character extraction processing program 8a according to this embodiment, the central processing unit 4 of the server device 1 mainly functions as an image data acquisition unit 4a, a print area determination unit 4b, and a character extraction unit 4c. The server device 1 (particularly the central processing unit 4) corresponds to an example of the "processing device" in the present invention, and also corresponds to an example of an entity that executes the "processing method" in the present invention.

[0034] The image data acquisition unit 4a acquires the contract image data 8b from the terminal device 2 operated by the user A and stores it in the auxiliary storage device 8. As described above, the contract image data 8b corresponds to the data generated by reading (scanning) the contract as a paper document using the scanner 20 connected to the terminal device 2 and output to the terminal device 2.

[0035] The stamp area determination unit 4b performs predetermined image processing on the contract image data 8b acquired by the image data acquisition unit 4a and stored in the auxiliary storage device 8 as described above, thereby determining a stamp area in the contract image corresponding to the area where the revenue stamp is affixed. As will be described in detail later, the stamp area determination unit 4b detects multiple feature points in the contract image using an image processing technique such as SIFT (Scale-Invariant Feature Transform), and performs a process of matching each of the detected feature points with each of multiple feature points in an image corresponding to an actual revenue stamp (hereinafter referred to as a "sample stamp image" as appropriate), thereby determining the stamp area. In this case, sample stamp image data 8c related to the sample stamp image is created in advance and stored in the auxiliary storage device 8, and the stamp area determination unit 4b reads the sample stamp image data 8c stored in the auxiliary storage device 8 and performs the above-mentioned process. This sample stamp image data 8c includes data related to sample stamp images corresponding to each of multiple types of revenue stamps, particularly data related to multiple feature points detected in advance for each of the multiple sample stamp images.

[0036] The character extraction unit 4c performs a process of extracting characters from areas in the contract image data 8b other than the stamp area determined by the stamp area determination unit 4b. Specifically, the character extraction unit 4c masks the stamp area in the contract image with white and extracts characters from the contract image after the masking process using OCR, thereby generating text data corresponding to the contract image data 8b. Note that, below, the process of determining the stamp area in the contract image and the process of masking the stamp area in the contract image will be collectively referred to as the "stamp removal process" where appropriate.

[0037] [Overall processing] Next, the flow of the overall process executed by the server device 1, including the stamp removal process according to the present embodiment, will be described with reference to Fig. 4. Fig. 4 is a flowchart showing an outline of the overall process executed by the server device 1 in the present embodiment. This overall process is realized by the central processing unit 4 of the server device 1 sequentially loading instructions and data (programs including the character extraction process program 8a described above) constituting a specific application stored in the auxiliary storage device 8 into the main storage device 5, and then performing operations on the loaded instructions and data.

[0038] First, in step S11, the central processing unit 4 of the server device 1 acquires the contract image data 8b stored in the auxiliary storage device 8. As described above, the contract image data 8b is generated by reading the contract with the scanner 20 connected to the terminal device 2, and is image data transmitted to the server device 1 via the terminal device 2.

[0039] Next, in step S12, the central processing unit 4 of the server device 1 determines the stamp area in the contract image corresponding to the area where the revenue stamp is affixed in the contract, and executes stamp removal processing to generate a contract image in which this stamp area has been masked with white. This stamp removal processing will be described in detail later with reference to Figure 5 etc.

[0040] Next, in step S13, the central processing unit 4 of the server device 1 executes a seal imprint removal process for removing the area in the contract where the seal imprint is applied (seal imprint area) from the contract image after the stamp removal process. Specifically, the central processing unit 4 determines, for example, the part in the contract image that is colored a specific color (typically vermilion) corresponding to the seal imprint as the seal imprint area, and generates a contract image in which this seal imprint area is masked with white. Note that while Figure 4 shows an example in which the seal imprint removal process is performed after the stamp removal process, in other examples, the stamp removal process may be performed after the seal imprint removal process.

[0041] Next, in step S14, the central processing unit 4 of the server device 1 generates text data of the contract by extracting characters using OCR from the contract image after the stamp removal process and the seal impression removal process. Then, in step S15, the central processing unit 4 reviews the contract based on the generated text data of the contract. Specifically, the central processing unit 4 analyzes the text data of the contract to generate review results including comments and ratings on the contract, and transmits data corresponding to these review results to the terminal device 2. Upon receiving the data corresponding to the review results, the terminal device 2 displays an image showing the review results on the display 2a. After step S15, the overall processing shown in FIG. 4 ends.

[0042] The server device 1 is not limited to reviewing contracts based on the text data of the contracts as described above. It may also manage contracts based on the text data. For example, the server device 1 may automatically extract contract information, such as the names of the parties, the contract's start date, end date, and whether or not the contract is automatically renewed, from the text data of the contract and register it in a database. This allows for the creation of contract lists suitable for various purposes, such as a list of contracts concluded with the same customer or a list sorted by contract expiration date. The server device 1 may also sort a list of multiple contracts by the contract's nearest renewal or end date. The server device 1 may also set contract viewing permissions for each organization or user. This allows for flexible configuration, such as contracts shared between the sales department and the management department, or contracts accessible only to the human resources department. The server device 1 may also impose IP address restrictions to restrict access to contract data from outside the company.

[0043] [Stamp removal process] Next, the stamp removal process according to this embodiment will be specifically described with reference to Figures 5 to 10. First, Figure 5 is a flowchart showing the stamp removal process executed by the server device 1 in this embodiment. As described above, this stamp removal process is executed by the central processing unit 4 of the server device 1 in step S12 of the overall process in Figure 4.

[0044] First, in step S21, the central processing unit 4 of the server device 1 acquires the sample stamp image data 8c stored in the auxiliary storage device 8. This sample stamp image data 8c includes data on sample stamp images corresponding to each of a plurality of types of revenue stamps, in particular data on a plurality of feature points detected in advance for each of these plurality of sample stamp images. Here, the sample stamp image data 8c used in this embodiment will be specifically described with reference to Fig. 6.

[0045] The left side of Figure 6 shows an example of a sample stamp image G1, specifically, multiple feature points detected by SIFT superimposed on the sample stamp image G1. These feature points are histogram data representing pattern information such as straight lines, curves, ridges, and bumps for each pixel of the image. The sample stamp image data 8c includes histogram data for each of the multiple feature points detected in the sample stamp image G1. The right side of Figure 6 shows an example of a histogram showing 128-dimensional feature values ​​(SIFT feature values) for one feature point. This histogram represents the gradient (change in brightness) of each direction at the feature point. SIFT, which uses such feature values, has the advantage of being able to accurately process the image even if the image being processed is tilted or the image size (including resolution) changes. This is because SIFT can maintain the shape of the histogram for each feature point by adjusting the histogram using the direction in which the brightness gradient is strongest at a feature point as the principal component (reference).

[0046] Here, by performing SIFT processing on a plurality of sample stamp images G1 corresponding to each of a plurality of types of revenue stamps, sample stamp image data 8c including data on feature points corresponding to each of the plurality of sample stamp images G1 is created in advance and stored in the auxiliary storage device 8. The sample stamp image G1 itself is an image of a revenue stamp published on the web (for example, a sample image) or an image obtained by scanning or photographing an actual revenue stamp. In addition, an image that covers a slightly wider range than the range corresponding to the size of an actual revenue stamp is used for the sample stamp image G1 (see FIG. 6).

[0047] Preferably, instead of using a large number of sample stamp images G1 corresponding to all types of revenue stamps, a small number of sample stamp images G1 corresponding to only a portion of types (for example, about four types) of revenue stamps may be used. This ensures the accuracy of the matching process described below and reduces the processing time.

[0048] In a more preferred example, for revenue stamps with the same revenue stamp design (here, "design" includes not only the content of the picture, such as flowers, drawn on the revenue stamp, but also the shape, position, and size of the characters "Revenue Stamp" and "Japanese Government" (excluding the amount of money) on the revenue stamp; the same applies below), only one sample stamp image G1 of any color and amount of revenue stamp may be used. In one example, since revenue stamps of the same format tend to have the same design, for revenue stamps of the same format, only one or two sample stamp images G1 of revenue stamps may be used, and mainly four or so sample stamp images G1 of different formats of revenue stamps may be used.

[0049] This is because the image on the revenue stamp is important in the matching process, and the parts other than the image on the revenue stamp (such as the color and amount of the revenue stamp) have little effect on the matching process. However, the amount of the revenue stamp tends to be difficult to match accurately, so this is dealt with separately using predetermined image processing (described in detail later with reference to Figure 9).

[0050] Next, in step S22 of Fig. 5, the central processing unit 4 of the server device 1 performs SIFT processing on the contract image data 8b acquired in step S11 of Fig. 4 in the same manner as the sample stamp image described above, thereby detecting feature points in the contract image. In this case, the central processing unit 4 obtains SIFT feature amounts (represented by a histogram) of the feature points in the contract image. Here, in this embodiment, the central processing unit 4 detects feature points only from images in the contract image that correspond to specific partial areas in the original contract. With reference to Fig. 7, the specific partial areas in the contract, from which feature points are detected in the contract image in this embodiment, will be specifically described.

[0051] As shown in FIG. 7, the central processing unit 4 detects feature points only from the area of ​​the contract image corresponding to the upper area R1 of the first page of the contract (for example, an area occupying the upper 1 / 3 of the first page). This area R1 is usually an area that includes the portion where the revenue stamp is affixed to the contract. Therefore, in this embodiment, the area where feature points are detected is limited to the area of ​​the contract image that corresponds to the area R1 where the revenue stamp is affixed to the contract. This makes it possible to ensure the accuracy of the matching process and reduce the processing time. The reason for this, which will be described in detail later, is that it is possible to reduce the number of feature points used in the matching process.

[0052] Additionally, in this embodiment, the central processing unit 4 detects feature points only in the colored portions of the contract image, specifically, in the colored portions of the contract image corresponding to the above-mentioned region R1. More specifically, the central processing unit 4 detects feature points in a contract image in which the black portions of the contract image, i.e., the non-colored portions, are masked with white. In this manner, the black text portions of the contract image are masked with white, while the revenue stamp portions of the contract image are not masked because they are colored. As a result, feature point detection is not performed in the white portions of the contract image, so the region in which feature point detection is performed can be accurately limited to the revenue stamp portions of the contract image. (Note that if there is colored text in the contract image, it is not masked, so feature point detection is also performed on this text portion.) This makes it possible to more effectively ensure the accuracy of the matching process and reduce processing time.

[0053] Next, in step S23 of Fig. 5, the central processing unit 4 of the server device 1 performs a matching process between the feature points of the contract image detected in step S22 and the feature points of the sample stamp image by continuing the SIFT process. Here, the matching process according to this embodiment will be specifically described with reference to Fig. 8.

[0054] FIG. 8 shows an example of a sample stamp image G1 on the right side and an example of a portion of a contract image G2 including an image corresponding to a revenue stamp on the left side. In particular, FIG. 8 shows multiple feature points detected by SIFT superimposed on each of the sample stamp image G1 and the contract image G2, and also shows pairs of matched feature points in the sample stamp image G1 and the contract image G2 connected by lines. The central processing unit 4 of the server device 1 detects feature points in the contract image G2 to obtain SIFT features, and then compares these SIFT features with each of the SIFT features of the multiple feature points in the sample stamp image G1. As a result, the central processing unit 4 matches the feature points detected in the contract image G2 with one of the multiple feature points in the sample stamp image G1, i.e., creates pairs of feature points in the contract image G2 and the sample stamp image G1. Typically, as shown in the bottom of FIG. 8, the central processing unit 4 creates pairs of feature points between the sample stamp image G1 and the contract image G2, the feature points having the closest SIFT feature amounts.

[0055] Next, in step S24 of Fig. 5, the central processing unit 4 of the server device 1 determines whether or not the matching process with the feature points of the sample stamp image G1 has been completed for all feature points in the contract image G2. As a result, if the matching process with all feature points has been completed (step S24: Yes), the central processing unit 4 proceeds to step S25. On the other hand, if the matching process with all feature points has not been completed (step S24: No), the central processing unit 4 returns to step S22. In this case, the central processing unit 4 repeats the processes of steps S22 to S24 until the matching process with all feature points has been completed. In this way, the central processing unit 4 detects all feature points in the area of ​​the contract image corresponding to the above-mentioned area R1, and performs the matching process with the feature points of the sample stamp image G1 for each of these feature points.

[0056] Next, in step S25, the central processing unit 4 of the server device 1 identifies a temporary area (temporary stamp area) within the contract image G2 corresponding to the revenue stamp based on the multiple feature points of the contract image G2 matched with the feature points of the sample stamp image G1, i.e., all feature points of the contract image G2 that have been paired with the feature points of the sample stamp image G1. Specifically, the central processing unit 4 sets a rectangular area that covers all of the multiple feature points of the matched contract image G2 as the temporary stamp area. In this case, the central processing unit 4 sets the rectangular temporary stamp area based on the ordinate of the topmost feature point, the ordinate of the bottommost feature point, the abscissa of the leftmost feature point, and the abscissa of the rightmost feature point among the multiple feature points of the matched contract image G2.

[0057] In specifying the temporary stamp area in this way, the central processing unit 4 does not use all of the feature points of the contract image G2 that have been matched with the feature points of the sample stamp image G1, but rather uses an algorithm such as RANSAC (Random Sample Consensus) or LMedS (Least Median of Squares) to extract feature points that have not been correctly matched from the feature points of the matched contract image G2, and specifies the temporary stamp area from the feature points excluding the extracted feature points. This makes it possible to set a temporary stamp area of ​​an appropriate range (a temporary stamp area whose range is accurately narrowed by correctly matched feature points) using only the feature points of the contract image G2 that have been correctly matched.

[0058] Next, in step S26, the central processing unit 4 of the server device 1 determines the final applicable stamp area by enlarging the provisional stamp area identified in step S25 by a magnification corresponding to the size corresponding to the actual revenue stamp. Here, with reference to Fig. 9, a method for determining the stamp area by enlarging the provisional stamp area in this embodiment will be specifically described.

[0059] FIG. 9 shows the same sample stamp image G1 and contract image G2 as in FIG. 8 (including multiple feature points and lines connecting the matched feature points). Also in FIG. 9, area R21 in the contract image G2 indicates the temporary stamp area, and area R31 in the sample stamp image G1 indicates the area in the sample stamp image G1 that corresponds to the temporary stamp area R21. Specifically, area R31 is a rectangular area that covers all of the multiple feature points in the sample stamp image G1 that were used to match the feature points in the contract image G2. Also in FIG. 9, area R22 in the contract image G2 indicates the stamp area to be finally applied, and area R32 in the sample stamp image G1 is an area that exactly corresponds to the size of the sample stamp image G1.

[0060] In this embodiment, the central processing unit 4 determines the stamp area R22 to be finally applied by enlarging the temporary stamp area R21 by a magnification corresponding to the size of the area R32 corresponding to the sample stamp image G1. Specifically, first, the central processing unit 4 calculates magnifications (for example, vertical and horizontal magnification ratios) for enlarging the area R31 corresponding to the temporary stamp area R21 in the sample stamp image G1 so that the size of the area R31 corresponding to the temporary stamp area R21 matches the size of the area R32 corresponding to the sample stamp image G1. Then, the central processing unit 4 determines the stamp area R22 by enlarging the temporary stamp area R21 by the magnifications thus calculated. In this case, it is preferable to determine the stamp area R22 by enlarging the temporary stamp area R21 in consideration of the positional relationship of the area R31 with respect to the area R32 in the sample stamp image G1.

[0061] By enlarging the provisional stamp region R21 in this way, a stamp region R22 having an aspect ratio similar to that of the sample stamp image G1 (corresponding to the aspect ratio of the actual revenue stamp) is determined. As a result, the problem of inability to accurately match the amount portion of the revenue stamp as described above can be appropriately solved. That is, even if the amount portion of the revenue stamp cannot be accurately matched, in other words, even if an area of ​​the revenue stamp lacking the amount portion is identified as the provisional stamp region R21, it is possible to determine a stamp region R22 that accurately covers the amount portion of the revenue stamp by enlarging this provisional stamp region R21 according to the size of the sample stamp image G1.

[0062] Next, in step S27 of Fig. 5, the central processing unit 4 of the server device 1 masks the stamp area R22 determined in step S26 in white for the contract image G2. Here, Fig. 10 shows an example of a portion of the contract image G3 after this masking process (a portion of the contract image that covers the stamp area). Specifically, this contract image G3 is obtained by masking the stamp area of ​​the original contract image G2 in white (see dashed line area R4). After step S27, the stamp removal process shown in Fig. 5 ends.

[0063] Next, referring to FIG. 11 , an example of characters extracted by OCR from a contract with a revenue stamp affixed thereto after the stamp removal process and the seal impression removal process according to this embodiment is shown. The top portion of FIG. 11 illustrates a portion of a contract with a revenue stamp affixed thereto. When OCR is performed on a contract image corresponding to such a contract after the stamp removal process according to this embodiment, the characters "Revenue Stamp," "Japanese Government," and "10,000 yen" from the revenue stamp are not extracted, but the characters "Construction Work Contract," which is the contract title, and the characters "●● (hereinafter referred to as the "client") and ●● (hereinafter referred to as the "contractor")," which are the contents of the contract, for example, are extracted. As a result, for example, if the topmost character string (corresponding to the characters at the top of the document) of the character string extracted by OCR is automatically set as the title of the text data of the document, it is possible to appropriately set only "Construction Work Contract" as the title of the text data.

[0064] [Action and effect] Next, the operation and effect of this embodiment will be described. As described above, according to the character extraction processing program 8a and the like according to this embodiment, in the server device 1, the image data acquisition unit 4a acquires the contract image data 8b, the stamp area determination unit 4b performs predetermined image processing (SIFT, etc.) on this contract image data 8b to determine the stamp area R22 in the contract image data 8b corresponding to the area where the revenue stamp is affixed in the contract, and the character extraction unit 4c performs processing to extract characters from areas other than the stamp area R22 in the contract image data 8b. This makes it possible to appropriately prevent the extraction of characters in the revenue stamp by OCR for a contract to which a revenue stamp is affixed. Therefore, according to this embodiment, even for a contract to which a revenue stamp is affixed, the processing to extract characters from the contract image data 8b can be performed accurately.

[0065] Furthermore, according to this embodiment, the stamp area determination unit 4b determines the stamp area R22 by performing image processing only on image data corresponding to a specific portion of the contract image data 8b, the area R1. This limits the area in the contract image data 8b where image processing is performed, rather than performing image processing on all areas of the contract image data 8b. This ensures the accuracy of image processing and reduces the processing time for image processing.

[0066] Furthermore, according to this embodiment, a contract consists of one or more pages, and the specific partial area is the area R1 at the top of the first page of the contract. Because revenue stamps tend to be affixed within the area R1 at the top of the first page of a contract, even if the area to be image processed in the contract image data 8b is limited to a partial area of ​​the contract, the stamp area R22 can be accurately extracted from the contract image data 8b.

[0067] Furthermore, according to this embodiment, the stamp area determination unit 4b determines the stamp area R22 by performing image processing only on the colored portions of the contract image data 8b. In other words, the stamp area determination unit 4b does not perform image processing on the black portions (nor the white portions) of the contract image data 8b. In such a case, image processing is not performed on the black text portions of the contract image data 8b, but since the revenue stamps are colored, image processing is performed on the revenue stamp portions of the contract image data 8b. This makes it possible to accurately limit the portions of the contract image data 8b to be image processed; in other words, it is possible to limit the area to which the revenue stamps are affixed. As a result, the stamp area R22 can be accurately extracted from the contract image data 8b while ensuring image processing accuracy and reducing processing time.

[0068] Furthermore, according to this embodiment, the stamp area determination unit 4b performs image processing on the contract image data 8b, including a process of identifying a provisional stamp area R21 corresponding to the revenue stamp within the contract image data 8b and a process of enlarging the provisional stamp area R21 by a magnification corresponding to the size corresponding to the actual revenue stamp, thereby determining the stamp area R22. As a result, even if the initially identified provisional stamp area R21 does not cover the entire revenue stamp, it is possible to accurately determine a stamp area R22 that covers the entire revenue stamp by enlarging this provisional stamp area R21 according to the size corresponding to the actual revenue stamp.

[0069] Furthermore, according to this embodiment, the character extraction unit 4c extracts characters from the area other than the stamp area R22 in the contract image data 8b by masking the stamp area R22 in the contract image data 8b. This makes it possible to accurately exclude the stamp area R22 from the contract image data 8b and extract characters.

[0070] Furthermore, according to this embodiment, the stamp area determination unit 4b performs image processing on the contract image data 8b by detecting feature points in the contract image data 8b and performing processing to match the detected feature points with feature points of the sample stamp image G1 corresponding to the actual revenue stamp, thereby determining the stamp area R22. This makes it possible to extract the stamp area R22 from the contract image data 8b with high accuracy.

[0071] [Variations] In the above embodiment, an example was shown in which the stamp area R22 in the contract image data 8b was determined using the SIFT image processing technique, but the use of SIFT is not limited to this. In other examples, the stamp area R22 in the contract image data 8b may be determined using SURF (Speeded Up Robust Features), ORB FAST Brief (Oriented FAST and Rotated BRIEF), AKAZE (Accelerated-KAZE), or the like.

[0072] Furthermore, in the above embodiment, image processing was performed only on the image data corresponding to the contract area R1 in the contract image data 8b, and image processing was performed only on the color parts in the contract image data 8b, but in other examples, image processing may be performed on the entire area of ​​the contract image data 8b, or image processing may be performed on parts other than color in the contract image data 8b.

[0073] Furthermore, in the above-described embodiment, the character extraction processing program 8a is installed on the server device 1, and the server device 1 extracts characters from the contract image data 8b. However, in another example, the present invention may be configured so that the character extraction processing program 8a is installed on a standalone computer. In this example, the terminal device 2 shown in FIG. 2 typically executes the character extraction processing program 8a installed therein as a standalone program, thereby realizing the processing device of the present invention. In this case, the terminal device 2 executes the processes shown in FIGS. 4 and 5. In yet another example, the character extraction processing program 8a (or a part of the program) may be installed in each of the server device 1 and the terminal device 2, and the server device 1 and the terminal device 2 may cooperate to realize the processing device of the present invention. In this case, the server device 1 and the terminal device 2 cooperate to execute the processes shown in FIGS. 4 and 5. In yet another example, the character extraction processing program 8a may be incorporated into dedicated hardware to realize the processing device of the present invention.

[0074] Furthermore, in the above-described embodiment, an example of applying the present invention to a contract is shown, but the present invention can be applied to various documents to which revenue stamps are affixed (for example, receipts and other receipts other than contracts).

[0075] The processes and procedures described herein can be realized not only by those explicitly described in the embodiments, but also by software, hardware, or a combination thereof. Specifically, the processes and procedures described herein can be realized by implementing logic corresponding to the processes in media such as integrated circuits, volatile memory, nonvolatile memory, magnetic disks, and optical storage. Furthermore, the processes and procedures described herein can be implemented as computer programs and executed by various computers, including terminal devices and server devices.

[0076] Although processes and procedures described herein are described as being performed by a single device, software, component, or module, such processes or procedures may be performed by multiple devices, multiple software, multiple components, and / or multiple modules. Furthermore, although various information described herein is described as being stored in a single memory or storage unit, such information may be stored in multiple memories within a single device or multiple memories distributed across multiple devices. Furthermore, software and hardware elements described herein may be realized by integrating them into fewer components or by decomposing them into more components. [Explanation of symbols]

[0077] 1. Server device (computer) 2. Terminal Device 3 Network 4. Central Processing Unit 4a Image data acquisition unit 4b Print area determination unit 4c Character extraction part 5 Main memory 8 Auxiliary storage 8a Character extraction processing program 8b Contract image data 8c Sample stamp image data 20 Scanner

Claims

1. A processing program executed by a computer to perform processing to extract characters from image data, The computer an image data acquisition unit that acquires image data of a predetermined document; a stamp area determination unit that performs predetermined image processing on the image data to determine a stamp area in the image data that corresponds to an area on the document to which a revenue stamp is affixed; and a character extraction unit that extracts characters from an area other than the print area in the image data; A processing program characterized by causing the program to function as follows.

2. The processing program according to claim 1 , wherein the stamp area determination unit determines the stamp area by performing the image processing only on image data of the document that corresponds to a specific partial area of ​​the document.

3. 3. The processing program according to claim 2, wherein the document is made up of one or more pages, and the specific partial area is an upper area of ​​a first page of the document.

4. The processing program according to claim 1 , wherein the print area determination unit determines the print area by performing the image processing only on a color portion of the image data.

5. A processing program according to any one of claims 1 to 4, wherein the stamp area determination unit determines the stamp area by performing the image processing on the image data by identifying a virtual area within the image data corresponding to the revenue stamp and enlarging the virtual area by a magnification corresponding to a size corresponding to an actual revenue stamp.

6. The processing program according to claim 1 , wherein the character extraction unit extracts characters from an area other than the stamp area in the image data by masking the stamp area in the image data.

7. A processing program according to any one of claims 1 to 6, wherein the stamp area determination unit determines the stamp area by performing image processing on the image data, which includes detecting feature points in the image data and matching the detected feature points with feature points of an image corresponding to an actual revenue stamp.

8. The processing program according to claim 1 , wherein the document is a contract.

9. A processing device that performs processing to extract characters from image data, an image data acquisition unit that acquires image data of a predetermined document; a stamp area determination unit that performs predetermined image processing on the image data to determine a stamp area in the image data that corresponds to an area on the document to which a revenue stamp is affixed; a character extraction unit that performs processing to extract characters from an area other than the print area in the image data; A processing device comprising:

10. A processing method executed by a computer to perform processing to extract characters from image data, comprising: an image data acquisition step of acquiring image data of a predetermined document; a stamp area determination step of determining a stamp area in the image data corresponding to an area on the document to which a revenue stamp is affixed by performing predetermined image processing on the image data; a character extraction step of extracting characters from an area other than the print area in the image data; A processing method comprising:

Images