Image processing device, image processing system, output device, image processing method, and image processing program

The image processing device generates composite seal images for machine learning to accurately recognize and remove seals from document images, ensuring precise character string extraction despite seal overlap.

JP2024158257APending Publication Date: 2024-11-08SHARP KK
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023073306
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-27
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

Existing image processing systems struggle to accurately recognize character strings in documents when seals overlap with the text, leading to misrecognition.

Method used

An image processing device generates a seal image and a pattern image in predetermined shading, combines them to create a composite seal image for machine learning, and uses a trained model to exclude seals from document images, enabling accurate character string recognition.

Benefits of technology

The system effectively recognizes and removes seal images from document images, allowing for precise extraction of character strings, even when seals overlap, thereby improving the accuracy of document image processing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024158257000001_ABST
    Figure 2024158257000001_ABST
Patent Text Reader

Abstract

To provide an image processing device capable of appropriately recognizing a character string of an input image including a seal image, an image processing system, an output device, an image processing method, and an image processing program.SOLUTION: An image processing device includes: a first generation processing unit for generating a seal image; a second generation processing unit for generating a pattern image represented by a predetermined shade; and a third generation processing unit for generating a composite seal image obtained by compositing the seal image generated by the first generation processing with a converted pattern image obtained by converting color components of the pattern image generated by the second generation processing unit as learning data to be used for machine learning.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a technique for performing image processing such as character recognition on an image. [Background technology]

[0002] Conventionally, there are known techniques for extracting character strings from images of documents such as documents, slips, etc. For example, there is known a technique for extracting handwritten input character strings character by character, setting a character frame (character rectangle) for each character, and correcting the inclination of the character frame based on a rotation angle of the character frame calculated using a reference line of the character frame (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2007-316754 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in documents such as documents and slips, a seal may be placed over a company name or an individual name. When a document image includes an image of a seal and the seal overlaps a character string, a problem occurs in that the character string overlapping the seal cannot be correctly recognized in the document image recognition process.

[0005] An object of the present disclosure is to provide an image processing device, an image processing system, an output device, an image processing method, and an image processing program capable of appropriately recognizing a character string in an input image including a seal image. [Means for solving the problem]

[0006] An image processing device according to one embodiment of the present disclosure includes a first generation processing unit that generates a seal image, a second generation processing unit that generates a pattern image expressed in a predetermined shade, and a third generation processing unit that generates a composite seal image by combining the seal image generated by the first generation processing unit and a converted pattern image obtained by converting the color components of the pattern image generated by the second generation processing unit, as learning data to be used for machine learning.

[0007] An image processing system according to another aspect of the present disclosure includes the image processing device and a learning device that generates a trained model by performing machine learning using the learning data generated by the image processing device.

[0008] An output device according to another aspect of the present disclosure uses the trained model generated by the learning device to perform a process of deleting a seal image and a process of recognizing a character string from an input image, and outputs the recognized character string.

[0009] An image processing method according to another aspect of the present disclosure is a method in which one or more processors execute the following steps: generate a seal image; generate a pattern image expressed in a predetermined shade; and generate a composite seal image by combining the seal image with a converted pattern image obtained by converting the color components of the pattern image, as learning data to be used for machine learning.

[0010] An image processing program relating to another aspect of the present disclosure is a program for causing one or more processors to execute the following steps: generate a seal image; generate a pattern image expressed in a predetermined shade; and generate a composite seal image by combining the seal image with a converted pattern image obtained by converting the color components of the pattern image, as learning data to be used for machine learning. Effect of the Invention

[0011] According to the present disclosure, it is possible to provide an image processing device, an image processing system, an output device, an image processing method, and an image processing program capable of appropriately recognizing a character string in an input image including a seal image. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a functional block diagram showing a configuration of an image processing device according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a diagram illustrating an example of a document (quote) according to an embodiment of the present disclosure. [Diagram 3] FIG. 3 is a diagram illustrating an example of a document image according to an embodiment of the present disclosure. [Figure 4A] FIG. 4A is a diagram illustrating an example of a seal image according to an embodiment of the present disclosure. [Figure 4B] FIG. 4B is a diagram illustrating an example of a seal image according to an embodiment of the present disclosure. [Figure 4C] FIG. 4C is a diagram illustrating an example of a seal image according to an embodiment of the present disclosure. [Figure 5A] FIG. 5A is a diagram illustrating an example of a pattern image according to an embodiment of the present disclosure. [Figure 5B] FIG. 5B is a diagram illustrating an example of a pattern image according to an embodiment of the present disclosure. [Figure 5C] FIG. 5C is a diagram illustrating an example of a pattern image according to an embodiment of the present disclosure. [Figure 5D] FIG. 5D is a diagram illustrating an example of a pattern image according to an embodiment of the present disclosure. [Figure 6] FIG. 6 is a flowchart showing an example of a procedure of image processing executed in the image processing device according to the embodiment of the present disclosure. [Figure 7A] FIG. 7A is a diagram illustrating an example of a method for generating a pattern image according to an embodiment of the present disclosure. [Figure 7B] FIG. 7B is a diagram illustrating an example of a method for generating a pattern image according to an embodiment of the present disclosure. [Figure 7C]FIG. 7C is a diagram illustrating an example of a method for generating a pattern image according to an embodiment of the present disclosure. [Figure 8A] FIG. 8A is a diagram illustrating an example of a method for generating a composite seal image according to an embodiment of the present disclosure. [Figure 8B] FIG. 8B is a diagram illustrating an example of a method for generating a composite seal image according to an embodiment of the present disclosure. [Figure 9A] FIG. 9A is a diagram illustrating another example of a method for generating a composite seal image according to an embodiment of the present disclosure. [Figure 9B] FIG. 9B is a diagram illustrating another example of a method for generating a composite seal image according to an embodiment of the present disclosure. [Figure 9C] FIG. 9C is a diagram illustrating another example of a method for generating a composite seal image according to an embodiment of the present disclosure. [Figure 10] FIG. 10 is a diagram showing another example of a method for generating a composite seal image according to an embodiment of the present disclosure. [Figure 11] FIG. 11 is a diagram illustrating an example of a method for detecting a region of a seal image according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that the following embodiments are examples of the present disclosure and are not intended to limit the technical scope of the present disclosure.

[0014] [Image processing device 1] 1 is a block diagram showing a configuration of an image processing device 1 according to an embodiment of the present disclosure. The image processing device 1 executes a process of generating data of a seal image or data in which the seal image is added to a document image as learning data (teacher data) to be used for machine learning. The image processing device 1 also performs machine learning using the learning data to generate a trained model that recognizes seals and character strings from an input image.

[0015] 1, the image processing device 1 includes a control unit 11, a storage unit 12, an operation display unit 13, and a communication unit 14. The image processing device 1 may be one or more cloud servers, or one or more physical servers.

[0016] The communication unit 14 is a communication interface for connecting the image processing device 1 to a network N1 in a wired or wireless manner and for executing data communication with an external device via the network N1 in accordance with a predetermined communication protocol. The network N1 is formed of, for example, the Internet, a LAN, or the like.

[0017] The operation display unit 13 is a user interface that includes a display unit, such as a liquid crystal display or an organic EL display, that displays various information, and an operation unit, such as a mouse, a keyboard, or a touch panel, that accepts operations.

[0018] The storage unit 12 is a non-volatile storage unit such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory that stores various information. The storage unit 12 stores a control program such as an image processing program (an example of an image processing program of the present disclosure) for causing the control unit 11 to execute image processing (see FIG. 6) described below. For example, the image processing program is non-temporarily recorded on a computer-readable recording medium such as a CD or a DVD, and is read by a reading device (not shown) such as a CD drive or a DVD drive provided in the image processing device 1 and stored in the storage unit 12. The image processing program may be distributed from a cloud server and stored in the storage unit 12.

[0019] The storage unit 12 also stores image data (scanned data, etc.) of documents and the like acquired from external devices.

[0020] FIG. 2 shows an estimate as an example of a document. As shown in FIG. 2, the estimate includes a plurality of character strings, such as the document classification ("estimate"), the issue date, the destination, the contact information of the estimator, the estimated amount, the product name, the quantity, the standard price, the discount amount, the subtotal, the consumption tax, and the total amount. The estimate also includes a seal A1 that is stamped by the creator of the estimate. If the estimate is electronic data, the seal A1 may be an electronic seal. The user scans the estimate in an external device and uploads the image data (input image) to the image processing device 1. When the control unit 11 acquires the image data of the estimate, it stores the image data in the storage unit 12. As another embodiment, the control unit 11 may acquire a document file of the estimate created in an external device and store the document file in the storage unit 12.

[0021] The control unit 11 has control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various arithmetic operations. The ROM stores in advance control programs such as a BIOS and an OS for causing the CPU to execute various processes. The RAM stores various information and is used as a temporary storage memory (work area) for the various processes executed by the CPU. The control unit 11 controls the image processing device 1 by executing, by the CPU, various control programs that are stored in advance in the ROM or the storage unit 12.

[0022] Incidentally, as shown in Fig. 2, in documents such as documents and slips, a seal A1 may be stamped so as to overlap a company name or an individual name. In this way, when the seal A1 overlaps with a character string, a problem occurs in that the character string overlapping the seal A1 is not correctly recognized in the document image recognition process. For example, as shown in Fig. 3, when object recognition process is performed on an input image P1 of an estimate, a character string rectangle indicating the range of the character string is not properly set for the character string overlapping the seal image A2, and the character string may be erroneously recognized. In response to this, the image processing device 1 according to the present disclosure has a configuration that enables proper recognition of the character string of an input image including a seal image, as shown below.

[0023] Specifically, as shown in FIG. 1, the control unit 11 includes various processing units such as an acquisition processing unit 111, a seal image generation processing unit 112, a pattern image generation processing unit 113, a learning image generation processing unit 114, and a learning processing unit 115. The control unit 11 functions as the various processing units by executing various processes according to the image processing program. Some or all of the processing units included in the control unit 11 may be configured with electronic circuits. The image processing program may be a program for causing a plurality of processors to function as the various processing units. The seal image generation processing unit 112 is an example of a first generation processing unit of the present disclosure, the pattern image generation processing unit 113 is an example of a second generation processing unit of the present disclosure, and the learning image generation processing unit 114 is an example of a third generation processing unit of the present disclosure.

[0024] The acquisition processing unit 111 acquires an input image. Specifically, the acquisition processing unit 111 acquires an input image (input data) that is the original data of the learning data from an external device. For example, the acquisition processing unit 111 acquires image data of various document images such as the estimate shown in Fig. 2. The acquisition processing unit 111 acquires image data of various documents that include a seal and image data of various documents that do not include a seal.

[0025] The acquisition processing unit 111 also acquires image data of a seal. Specifically, the acquisition processing unit 111 acquires image data of seals of various shapes, colors, fonts, and names (company names, personal names, etc.).

[0026] The seal image generation processing unit 112 generates a seal image Ps. Specifically, the seal image generation processing unit 112 generates various seal images Ps based on image data of an actual seal acquired by the acquisition processing unit 111. Figures 4A to 4C are examples of the seal image Ps generated by the seal image generation processing unit 112. A specific method of generating the seal image Ps will be described later.

[0027] The pattern image generation processing unit 113 generates a pattern image Pt expressed in a predetermined shading. Specifically, the pattern image generation processing unit 113 generates a plurality of types of pattern images Pt having different shading patterns. Figures 5A to 5D are examples of the pattern images Pt generated by the pattern image generation processing unit 113. A specific method of generating the pattern image Pt will be described later.

[0028] The training image generation processing unit 114 generates training data used for machine learning. Specifically, the training image generation processing unit 114 generates a composite seal image as training data by combining the seal image Ps generated by the seal image generation processing unit 112 and the pattern image Pt generated by the pattern image generation processing unit 113. For example, the training image generation processing unit 114 generates a composite seal image (training data) by combining the seal image Ps and a pattern image Pt after converting (multiplying by a coefficient) the color components (tones) of the pattern image Pt. In addition, the training image generation processing unit 114 generates an entire image (an example of a composite document image of the present disclosure) by adding the generated composite seal image to a document image as training data. A specific method for generating the training data will be described later.

[0029] The learning processing unit 115 generates a trained model by performing machine learning using the training data (synthetic seal image, entire image) generated by the training image generation processing unit 114. The learning processing unit 115 may be configured as a device (learning device) separate from the image processing device 1. In this case, an image processing system may be constructed that includes the image processing device 1 and a learning device that generates a trained model by performing machine learning using the training data generated by the image processing device 1.

[0030] In addition, the trained model can be applied to various output devices (such as character recognition devices and general-purpose computers). For example, when an image to be character recognized is input to the output device, the control unit of the output device uses the trained model to execute a seal recognition process and a character recognition process on the input image and outputs the character recognition result. For example, when the control unit recognizes a seal image included in a document image, it excludes (deletes) the seal image from the document image. Then, the control unit recognizes a character string in the document image from which the seal image has been excluded by OCR processing or the like, and outputs the recognized character string as a character recognition result or registers it in a database. This makes it possible to appropriately extract characters of a character string from a document image including a seal.

[0031] Machine learning includes algorithms such as supervised learning using supervised data, unsupervised learning using unsupervised data, and reinforcement learning. Furthermore, to realize these methods, a method called "deep learning" is used, which learns to extract the feature amount itself. In this embodiment, the control unit 11 has a learning model based on the various algorithms described above. In this embodiment, data in which a synthetic impression image is added to a document image corresponds to supervised data, and data of the synthetic seal image and the document image corresponds to unsupervised data. The control unit 11 can perform machine learning using the supervised data and unsupervised data as input data (learning data) to generate a trained model that executes the seal recognition process and the character recognition process. That is, the image processing device 1 functions as a learning device that generates a trained model.

[0032] The trained model may be downloaded to an operation terminal for use, or may be stored in a server (cloud server) and accessed from an operation terminal via the Internet, etc. When an arbitrary input image is input to the operation terminal, the trained model outputs an optimal character recognition result.

[0033] The control unit 11 of the image processing device 1 may display various information on the operation display unit 13 in the process of generating learning data (a composite seal image, an entire image, etc.). For example, the control unit 11 displays on the operation screen each of the seal image Ps generated by the seal image generation processing unit 112, the pattern image Pt generated by the pattern image generation processing unit 113, and the composite seal image and the entire image generated by the learning image generation processing unit 114. This allows a user who generates learning data to generate the learning data while checking the generation process of the learning data.

[0034] The specific configuration of each processing unit will be described below with a specific example.

[0035] [Image processing] FIG. 6 is a flowchart showing an example of a procedure of image processing executed in the image processing device 1.

[0036] The present disclosure can be understood as an image processing method (image processing method of the present disclosure) that executes one or more steps included in the image processing. One or more steps included in the image processing described here may be omitted as appropriate. The steps in the image processing may be executed in a different order as long as the same action and effect is achieved. Furthermore, although an example is described here in which the control unit 11 of the image processing device 1 executes each step in the image processing, in other embodiments, one or more processors may execute each step in the image processing in a distributed manner.

[0037] In step S1, the control unit 11 executes a process of generating a seal image. Specifically, when the control unit 11 acquires image data of an actual seal, it generates various seal images Ps based on the acquired image data.

[0038] For example, the control unit 11 generates a seal image Ps by changing the characters of the seal based on an actual seal. For example, the control unit 11 determines the outer shape of the seal, such as a square seal or a round seal, and generates a seal image Ps by imaging the company name text using a font, such as an old seal type, a lucky font, or a seal script type, that is commonly used for seals. The control unit 11 also generates the seal image Ps by deforming the outer shape of the seal according to the number of characters so that the characters of the seal fit within the area. For example, in the case of a circular seal, the control unit 11 may make the outer periphery circular and mask the inside according to the area. Figures 4A to 4C are examples of seal images Ps generated as described above.

[0039] The control unit 11 may also generate a seal image with the same color. For example, the control unit 11 may set the color of the seal image Ps to the same red color, for example, fixed values ​​such as [R,G,B]=[255,0,0] or [224,0,0], or may assign random numbers within a predetermined range. For example, when assigning the value of R in the range of 224 to 255, the control unit 11 may specify [R,G,B]=[random(224,255),0,0] using a function that generates random numbers with a range specification. Similarly, the control unit 11 may set the width of the rounded rectangle or circle to a fixed value, or may specify it as a random number within a predetermined range. The control unit 11 may also specify the font as a fixed font, or may list the font file names and specify it as a random number according to the index value of the list. When the font has multiple typefaces (standard, bold, italic, etc.), the control unit 11 may specify the index value as a random number. Furthermore, the control unit 11 may set the pixel size of the seal image Ps to, for example, 320×320, or may specify it as a random number within a predetermined range (for example, a range of 200 to 350).

[0040] The control unit 11 generates one seal image Ps in step S1 and stores the generated seal image Ps in the storage unit 12. Note that the control unit 11 generates seal images Ps with different outer shapes, sizes, colors, fonts, etc., each time the control unit 11 executes the image processing shown in FIG.

[0041] Next, in step S2, the control unit 11 executes a process of generating a pattern image Pt (shade pattern image) expressed in a predetermined shade. Specifically, the control unit 11 generates a pattern image Pt in which the same shade and the same type of pattern are drawn uniformly. Note that the shade and pattern do not need to be completely uniform and identical, but may be roughly the same (similar or substantially the same). Furthermore, the control unit 11 generates a pattern image Pt with a different shade pattern each time it executes the image processing shown in FIG. 6.

[0042] For example, the control unit 11 generates one of the pattern images Pt (see Figs. 5A to 5D) of a gradation pattern in which random dot, line, or mosaic patterns are arranged. Fig. 5A shows an embossed image such as wallpaper, Figs. 5B and 5C show an error diffusion image of a predetermined gradation, and Fig. 5D shows an image of cracks. The pattern image Pt may be an image of a marble pattern. The control unit 11 also generates the pattern image Pt by grayscaling each image. The control unit 11 may generate a pattern image Pt in which the gradation is set to a certain range, for example, 50 to 100 gradations on a grayscale of 0 to 100 gradations.

[0043] After generating one pattern image Pt, the control unit 11 subsequently executes a process of changing the size (number of pixels in the vertical and horizontal directions) of the pattern image Pt. The control unit 11 changes the size of the pattern image Pt based on the size of the seal image Ps.

[0044] Specifically, the control unit 11 generates a pattern image Pt of the same size as the seal image Ps. For example, when the size of the seal image Ps is smaller than the size of the pattern image Pt, the control unit 11 reduces the size of the pattern image Pt as a first method. FIG. 7A shows the pattern image Pt and the seal image Ps. As shown in FIG. 7A, the control unit 11 aligns the upper left corner of the seal image Ps with the upper left corner of the pattern image Pt, and reduces the entire pattern image Pt so that the lower right corner of the pattern image Pt fits the lower right corner of the seal image Ps (see FIG. 7B).

[0045] In addition, the control unit 11 may rotate or flip the seal image Ps by 90 degrees x N (0, 1, 2, 3) in the case of rounded corners, or rotate it by any angle in the case of a circle, so that the seal image Ps is not in the same position relative to the pattern image Pt in other composite seal images (so that the position of the seal image Ps on the pattern image Pt is different for each composite seal image).

[0046] In addition, when the size of the seal image Ps is smaller than the size of the pattern image Pt, the control unit 11 may cut out an area of ​​the pattern image Pt having the same size as the seal image Ps as a second method (see FIG. 7C). Note that the control unit 11 may randomly determine the cut-out position within the range of the pattern image Pt so that the cut-out position is not the same as the position of the seal image Ps relative to the pattern image Pt in other composite seal images (so that the position where the seal image Ps is placed on the pattern image Pt is different for each composite seal image).

[0047] Furthermore, when the size of the seal image Ps is smaller than the size of the pattern image Pt, the control unit 11 may determine the occurrence probability of the first method and the second method to be an arbitrary probability (for example, 1:1) as a third method, and change the size of the pattern image Pt. The control unit 11 may change the probability every time the image processing shown in Fig. 6 is executed. In this way, the control unit 11 may execute at least one of the processing of the first method and the second method.

[0048] If the size of the seal image Ps is larger than the size of the pattern image Pt, the control unit 11 enlarges the size of the pattern image Pt to match the size of the seal image Ps. If the size of the seal image Ps is the same as the size of the pattern image Pt, the control unit 11 omits the process of changing the size of the pattern image Pt.

[0049] The control unit 11 stores the pattern image Pt generated in step S2 in the storage unit 12. After generating the pattern image Pt in step S2, the control unit 11 shifts the process to step S3.

[0050] Next, in step S3, the control unit 11 combines the seal image Ps generated in step S1 with the pattern image Pt generated in step S2. Specifically, the control unit 11 combines the seal image Ps with the pattern image Pt after converting the color components of the pattern image Pt.

[0051] Fig. 8A shows the concept of the synthesis method in step S3, and Fig. 8B shows a specific example of the synthesis method. For example, as shown in Fig. 8B, first, the control unit 11 synthesizes an R component image (red component image) of the seal image Ps with an image (corresponding to the pattern image Pt after the color component of the pattern image Pt is converted) in which the gradation of the pattern image Pt is multiplied by a coefficient (KA times) to generate an R synthetic image, synthesizes a G component image (green component image) of the seal image Ps with an image in which the gradation of the pattern image Pt is multiplied by a coefficient (KA times) to generate a G synthetic image, and synthesizes a B component image (blue component image) of the seal image Ps with an image in which the gradation of the pattern image Pt is multiplied by a coefficient (KA times) to generate a B synthetic image. Next, the control unit 11 generates an image (G correction image) in which the gradation of the G synthetic image is multiplied by a coefficient (KG times). Finally, the control unit 11 synthesizes the R synthetic image, the G correction image, and the B synthetic image to generate a synthetic seal image.

[0052] In this way, in order to change the color of the composite seal image (output data), the control unit 11 multiplies the gradation of the pattern image Pt by KA for each color component image of the seal image Ps, performs an addition process with saturation processing, and further multiplies only the image of the green component (G channel) by KG. The coefficients KA and KG (parameters) are both random numbers with ranges to increase the overall variation in color. For example, the control unit 11 sets the coefficient KA to a value within the range of 1.0 to 1.1, and sets the coefficient KG to a value within the range of 0.9 to 1.1. The coefficient KA is an example of the first coefficient (first parameter) of the present disclosure, and the coefficient KG is an example of the second coefficient (second parameter) of the present disclosure.

[0053] The method of generating a composite seal image is not limited to the method shown in FIGS. 8A and 8B, and may be the following method.

[0054] As another method of generating a composite seal image, for example, as shown in FIG. 9A, the control unit 11 may multiply the gradation of the pattern image Pt by KA (for example, the coefficient KA is a value within a range of 0.5 to 1.5) and then subtract DA gradation (for example, 0 to 100 gradations). By increasing the subtraction value DA, the final pattern image Pt has darker shading. Also, by setting the coefficient KA to a value greater than 1, the noise component becomes stronger. FIG. 9B shows a pattern image Pt when the coefficient KA is set to 0.6 and the DA gradation is set to 10 gradations, and FIG. 9C shows a pattern image Pt when the coefficient KA is set to 1.5 and the DA gradation is set to 50 gradations.

[0055] The control unit 11 combines an image obtained by subtracting the DA gradation from the KA-multiplied pattern image Pt with a seal image Ps, and generates a composite seal image by combining an R composite image of the combined image, a G corrected image obtained by multiplying the G composite image of the combined image by KG, and a B composite image of the combined image.

[0056] As another method for generating a composite seal image, instead of multiplying the gradation of the pattern image Pt by KA and subtracting the DA gradation in the method shown in Fig. 9A, the pattern image Pt may be tone corrected (LUT conversion or calculation). For example, the control unit 11 converts the gradation of the pattern image Pt based on the gradation conversion characteristic (LUT conversion) shown in Fig. 10, synthesizes the pattern image Pt after the gradation conversion with the seal image Ps, and generates a composite seal image by synthesizing the R composite image of the synthesized image, the G corrected image obtained by multiplying the G composite image of the synthesized image by KG, and the B composite image of the synthesized image.

[0057] The control unit 11 generates one composite seal image in step S3, and stores the generated composite seal image in the storage unit 12. When the control unit 11 generates the composite seal image in step S3, it shifts the process to step S4.

[0058] Next, in step S4, the control unit 11 executes a process of rotating the composite seal image generated in step S3. Specifically, the control unit 11 rotates the image seal image by a predetermined angle by a linear transformation process such as a rotation matrix. For example, the control unit 11 may execute the rotation process using a Gaussian distribution (σ:3) of approximately ±10 degrees as an angle parameter.

[0059] Next, in step S5, the control unit 11 sets the seal area (rectangular frame area). Specifically, the control unit 11 sets a rectangular frame area according to the outer shape of the composite seal image as coordinate information. For example, if the seal is circular in shape, the seal area will not change even if the composite seal image is rotated. In contrast, if the seal is non-circular (e.g., a rectangle with rounded corners), the seal area will change according to the rotation angle of the composite seal image. The seal area is a rectangular frame area for recognizing the seal from an input image (such as a document image).

[0060] Here, it is possible to calculate the seal area from the parameters used when drawing the seal image, but when the parameters are unknown, such as when processing an existing seal, it is more efficient to generate a seal image and detect the seal area (rectangular frame area) from the seal image rather than analyzing the existing seal.

[0061] Therefore, as shown in FIG. 11, for example, the control unit 11 extracts edges of image data of blue component (B channel) (differential filter, Canny filter, etc.), calculates the sum (or average) of pixel values ​​in the vertical and horizontal directions, and detects points where a numerical value exists by searching from the maximum and minimum positions. The control unit 11 specifies the area where the pattern image exists based on the detected point, and adds the coordinate information of the specified area to the coordinate information when the seal image was drawn to obtain the coordinate value of the annotation information. If the coordinates of the upper left reference point of the seal image when the seal image was drawn are (x, y), the area of ​​the composite seal image (seal area) is expressed as (x+xs, y+ys) to (x+xe, y+ye). The control unit 11 adds the information of the seal area to the generated composite seal image.

[0062] Next, in step S6, the control unit 11 acquires a document image. Specifically, the control unit 11 acquires a document image input from an external device. The control unit 11 also acquires image data of various documents that do not include a seal. The control unit 11 may also acquire image data of various documents that include a seal.

[0063] Next, in step S7, the control unit 11 executes a process of synthesizing the generated composite seal image with the acquired document image. The composite seal image to be synthesized may be the image generated in step S3, or the image rotated in step S4. In addition, the seal area set in step S5 is added to the composite seal image to be synthesized.

[0064] Specifically, the control unit 11 sets a reference position (reference point) for arranging the composite seal image in the document image. For example, the control unit 11 sets the reference point by a random number within a range that does not extend outside the entire area of ​​the document image. In another embodiment, when the control unit 11 wants to arrange the composite seal image in a predetermined range in the upper right corner of the document image, it sets the reference point by a random number within the predetermined range.

[0065] The control unit 11 may also perform synthesis by subtracting the difference between the maximum value of the image data of the document image and the image data value of the synthetic seal image from the document image. The maximum value of the image data is, for example, 255 in the case of 8-bit data. This allows a synthetic image close to reality to be generated without impairing the color of the document image.

[0066] In this way, in step S7, the control unit 11 generates an entire image by adding the generated composite seal image to the document image. After step S7, the control unit 11 shifts the process to step S8.

[0067] In step S8, the control unit 11 outputs learning data. Specifically, the control unit 11 outputs the entire image generated in step S7 as learning data. In another embodiment, the control unit 11 may output the composite seal image generated in step S5 to which information on the seal area is added as learning data. The control unit 11 may also output the entire image and the composite seal image as learning data.

[0068] In the above image processing, the control unit 11 generates one seal image Ps and one pattern image Pt, and generates one composite seal image (whole image) based on the seal image Ps and the pattern image Pt. Each time the control unit 11 executes the above image processing, it generates one seal image Ps and one pattern image Pt to generate one composite seal image (whole image). The control unit 11 repeatedly executes the above image processing to generate multiple learning data (composite seal image, whole image).

[0069] The control unit 11 performs machine learning using multiple learning data (whole image, composite seal image) to generate a trained model that recognizes a seal from an input document image when a document image of an actual document (see FIG. 3) is input. The trained model may be a model that recognizes a seal from a document image and recognizes a character string from the document image excluding the recognized seal.

[0070] Furthermore, when the trained model is installed in a character recognition device (such as a user terminal), when an image to be character recognized (such as a document image) is input, the character recognition device uses the trained model to perform seal recognition processing and character recognition processing on the input image and outputs the character recognition result. For example, when the character recognition device recognizes a seal image included in a document image, it removes the seal image from the document image. The character recognition device then recognizes character strings in the document image from which the seal image has been removed by OCR processing or the like, and outputs the recognized character strings as a character recognition result or registers them in a database.

[0071] As described above, the image processing device 1 of this embodiment generates a seal image, generates a pattern image expressed in a predetermined shade, and generates a composite seal image by combining the seal image with a converted pattern image in which at least one of the multiple color components of the pattern image is converted, as learning data to be used for machine learning.

[0072] According to the above configuration, for example, it is possible to appropriately recognize a seal image from a document image, and therefore it is possible to remove the seal image from the document image. This makes it possible to generate learning data that can recognize a character string in a document image without being affected by the seal image. Therefore, it becomes possible to appropriately recognize a character string in an input image that includes a seal image.

[0073] The image processing device 1 also generates a first composite seal image by combining a first color component image of a first color component of the seal image Ps with a first pattern image obtained by multiplying the gradation of the pattern image Pt by a first coefficient. The image processing device 1 also generates a second composite seal image by multiplying the gradation of an image obtained by combining a second color component image of a second color component of the seal image Ps with the first pattern image by a second coefficient. The image processing device 1 then generates a composite seal image by combining the first composite seal image with the second composite seal image. The first color component is, for example, a red component or a blue component, and the second color component is a green component.

[0074] As described above, in the past, by simply synthesizing an electronic seal with an image, there was little variation in color and blurring compared to the actual seal image, making it difficult to generate learning data with sufficient accuracy. In addition, when synthesizing images, pixel replacement or overwriting results in a large difference from the actual seal image. In response to this, the image processing device 1 according to this embodiment is configured to add a change selected from several shading data to the color data of the uniform seal part, change the color, change the size and angle, and add the image to be added by subtracting the difference value from the white value of the target image. In addition, the image processing device 1 is configured to add a shading change selected from several shading data to seal data generated with the same uniform color, change at least one color of RGB, change the size of the pattern image within a range according to the shape of the seal image, and change the angle of the seal image according to a Gaussian distribution. In addition, the image processing device 1 is configured to add image data to which a seal image is to be added by subtracting the difference amount from the numerical value indicating the white of the image data to which the seal image is to be added.

[0075] [Disclosure Note] The following will provide an overview of the disclosure extracted from the above-described embodiment. Note that the configurations and processing functions described in the following supplementary notes can be selected and combined as desired.

[0076] <Appendix 1> A first generation processing unit that generates a seal image; A second generation processing unit that generates a pattern image expressed in a predetermined shade; a third generation processing unit that generates a composite seal image by combining the seal image generated by the first generation processing unit and a pattern image after converting a color component of the pattern image generated by the second generation processing unit, as learning data to be used for machine learning; An image processing device comprising:

[0077] <Appendix 2> The third generation processing unit generates a composite document image by combining the composite seal image and a predetermined document image as learning data. 2. An image processing device as described in claim 1.

[0078] <Appendix 3> The first generation processing unit generates the seal image in the same uniform color. 3. An image processing device as described in claim 2.

[0079] <Appendix 4> The second generation processing unit generates the pattern image having a pattern of the same uniform shading and type. 4. The image processing device according to claim 2 or 3.

[0080] <Appendix 5> The second generation processing unit executes at least one of a process of changing a size of the pattern image based on a size of the seal image, and a process of cutting out an area of ​​the pattern image having the same size as the seal image. 5. An image processing device as described in claim 4.

[0081] <Appendix 6> The third generation processing unit is a first composite seal image obtained by combining a first color component image of the first color component of the seal image and a first pattern image obtained by multiplying the gradation of the pattern image by a first coefficient; a second composite seal image obtained by multiplying the gradation of an image obtained by combining a second color component image of the second color component of the seal image and the first pattern image by a second coefficient; and generating the composite seal image by combining the above. 6. The image processing device according to claim 1,

[0082] <Appendix 7> The first color component is a red component or a blue component, and the second color component is a green component. 7. The image processing device according to claim 6.

[0083] <Appendix 8> An image processing device according to any one of claims 1 to 7, A learning device that generates a trained model by performing machine learning using the learning data generated by the image processing device; An image processing system comprising:

[0084] <Appendix 9> An output device that uses the trained model generated by the learning device described in appendix 8 to perform a process of deleting a seal image and a process of recognizing a character string from an input image, and outputs the recognized character string. [Explanation of symbols]

[0085] 1: Image processing device 11: Control section 12: Storage section 111: Acquisition processing unit 112: Seal image generation processing unit 113: Pattern image generation processing unit 114: Learning image generation processing unit 115: Learning processing unit P1: Input image Ps:Seal image Pt: Pattern image KA: Coefficient (parameter) KG: Coefficient (parameter)

Claims

1. A first generation processing unit that generates a seal image; A second generation processing unit that generates a pattern image expressed in a predetermined shade; a third generation processing unit that generates a composite seal image by combining the seal image generated by the first generation processing unit and a pattern image after converting a color component of the pattern image generated by the second generation processing unit, as learning data to be used for machine learning; An image processing device comprising:

2. The third generation processing unit generates a composite document image by combining the composite seal image and a predetermined document image as learning data. The image processing device according to claim 1 .

3. The first generation processing unit generates the seal image in the same uniform color. The image processing device according to claim 2 .

4. the second generation processing unit generates the pattern image having a pattern of a uniformly similar shade and type. The image processing device according to claim 2 .

5. The second generation processing unit executes at least one of a process of changing a size of the pattern image based on a size of the seal image, and a process of cutting out an area of ​​the pattern image having the same size as the seal image. The image processing device according to claim 4.

6. The third generation processing unit is a first composite seal image obtained by combining a first color component image of the first color component of the seal image and a first pattern image obtained by multiplying the gradation of the pattern image by a first coefficient; a second composite seal image obtained by multiplying the gradation of an image obtained by combining a second color component image of the second color component of the seal image and the first pattern image by a second coefficient; and generating the composite seal image by combining the above. The image processing device according to claim 1 .

7. the first color component is a red component or a blue component, and the second color component is a green component; The image processing device according to claim 6.

8. An image processing device according to any one of claims 1 to 7, A learning device that generates a trained model by performing machine learning using the learning data generated by the image processing device; An image processing system comprising:

9. An output device that uses the trained model generated by the learning device according to claim 8 to execute a process of deleting a seal image and a process of recognizing a character string from an input image, and outputs the recognized character string.

10. generating a seal image; generating a pattern image expressed in a predetermined shade; generating a composite seal image by combining the seal image and a pattern image obtained by converting a color component of the pattern image, as learning data to be used in machine learning; An image processing method executed by one or more processors.

11. generating a seal image; generating a pattern image expressed in a predetermined shade; generating a composite seal image by combining the seal image and a pattern image obtained by converting a color component of the pattern image, as learning data to be used in machine learning; An image processing program for causing one or more processors to execute the above.

Citation Information

Patent Citations

  • Handwritten character processing device and method

    JP2007316754A