Information processing system, information processing method, and information processing program

Abstract

This invention provides an information processing system, an information processing method, and an information processing program that can improve character recognition accuracy by appropriately rendering document data. [Solution] The image processing device 1 includes an extraction processing unit 112 that extracts target objects, at least one of text objects and drawing objects, from document data; a replacement processing unit 113, a deletion processing unit 114, and a modification processing unit 115 that perform predetermined correction processing on the target objects when the target objects extracted by the extraction processing unit 112 match pre-registered target objects; and a rendering processing unit 116 that performs rendering processing on the document data in which the target objects have been corrected to generate image data for character recognition.

Description

Technical Field

[0001] The present disclosure relates to a technique for performing image processing such as character recognition on an input image.

Background Art

[0002] Conventionally, a technique for character recognition (OCR processing) of document data such as forms has been known. For example, a technique is known in which a character object is discriminated from image data after rendering PDL data (page description language), and character recognition processing is performed on the discriminated character object to extract character code information (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional technique, for example, when rendering (converting to an image) document data (document file) such as PDF, if a character code that does not correspond to a font that can be used in the conversion process is included, characters may be garbled, or in the OCR processing after the conversion process, a shadow image may overlap the characters, making it difficult to recognize the characters, or it may be difficult to recognize the characters due to the influence of the background base, etc., and a problem of deterioration in character recognition accuracy occurs.

[0005] An object of the present disclosure is to provide an information processing system, an information processing method, and an information processing program capable of improving character recognition accuracy by appropriately rendering document data.

Means for Solving the Problems

[0006] An information processing system according to one aspect of this disclosure comprises an extraction processing unit, a correction processing unit, and a generation processing unit. The extraction processing unit extracts at least one target object from document data, consisting of text objects and drawing objects. The correction processing unit performs a predetermined correction process on the target object if the target object extracted by the extraction processing unit matches a pre-registered object to be processed. The generation processing unit performs a rendering process on the document data from which the target object has been corrected by the correction processing unit to generate image data for character recognition.

[0007] An information processing method relating to another aspect of this disclosure is a method in which one or more processors perform the following steps: extract target objects from document data, at least one of text objects and drawing objects; perform a predetermined correction process on the target object if the target object matches a pre-registered object to be processed; and perform a rendering process on the document data in which the target object has been corrected to generate image data for character recognition.

[0008] An information processing program relating to another aspect of this disclosure is a program that causes one or more processors to perform the following actions: extract target objects from document data, at least one of text objects and drawing objects; perform a predetermined correction process on the target object if the target object matches a pre-registered object to be processed; and perform a rendering process on the document data in which the target object has been corrected to generate image data for character recognition. [Effects of the Invention]

[0009] According to this disclosure, it is possible to provide an information processing system, an information processing method, and an information processing program that can improve character recognition accuracy by appropriately rendering document data. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 is a functional block diagram showing the configuration of an image processing system according to an embodiment of this disclosure. [Figure 2] Figure 2 shows an example of a document (PDF file) according to an embodiment of this disclosure. [Figure 3] Figure 3 is a flowchart showing an example of the procedure for character extraction processing performed in the image processing apparatus according to the embodiment of this disclosure. [Figure 4] Figure 4 shows a specific example of a replacement character list used in an image processing apparatus according to the embodiment of this disclosure. [Figure 5] Figure 5 is a flowchart showing an example of the procedure for specific character replacement processing performed in the image processing apparatus according to the embodiment of this disclosure. [Figure 6] Figure 6 is a flowchart showing an example of the procedure for specific image deletion processing performed in the image processing apparatus according to the embodiment of this disclosure. [Figure 7A] Figure 7A shows a specific example of a seal impression image included in an image object according to the embodiment of this disclosure. [Figure 7B] Figure 7B shows a specific example of a seal impression image included in an image object according to the embodiment of this disclosure. [Figure 7C] Figure 7C shows a specific example of a seal impression image included in an image object according to the embodiment of this disclosure. [Figure 7D] Figure 7D shows a specific example of a seal impression image included in an image object according to the embodiment of this disclosure. [Figure 8] Figure 8 is a flowchart showing an example of the procedure for background surface modification processing performed in an image processing apparatus according to the embodiment of this disclosure. [Figure 9A] Figure 9A shows a specific example of a table that is subject to the background surface modification process according to the embodiment of this disclosure. [Figure 9B]FIG. 9B is a diagram showing a modified example of the background background of a table according to an embodiment of the present disclosure. [Figure 9C] FIG. 9C is a diagram showing a modified example of the background background of a table according to an embodiment of the present disclosure. [Figure 10A] FIG. 10A is a diagram showing specific examples of characters and backgrounds to be subjected to background background change processing according to an embodiment of the present disclosure. [Figure 10B] FIG. 10B is a diagram showing a modified example of characters and backgrounds to be subjected to background background change processing according to an embodiment of the present disclosure. [Figure 10C] FIG. 10C is a diagram showing a modified example of characters and backgrounds to be subjected to background background change processing according to an embodiment of the present disclosure.

Mode for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. The following embodiments are an example of embodying the present disclosure and do not have the character of limiting the technical scope of the present disclosure.

[0012] FIG. 1 is a block diagram showing the configuration of an image processing system 10 according to an embodiment of the present disclosure. The image processing system 10 includes an image processing apparatus 1 and an operation terminal 2. The image processing apparatus 1 and the operation terminal 2 are connected to each other via a network N1 (for example, the Internet, LAN, etc.). The image processing system 10 may include a plurality of operation terminals 2.

[0013] In the image processing system 10, the image processing apparatus 1 acquires document data (or document image data) such as forms transmitted from the operation terminal 2, and extracts a desired character string (a character string to be managed) from the document data. For example, the operation terminal 2 scans paper-form documents such as invoices, estimates, delivery notes, orders, receipts, receipts, and other documents, and transmits the generated document image data (such as a PDF file) to the image processing apparatus 1. Further, the operation terminal 2 creates a document file of the form based on the user's operation by, for example, a document creation application, and transmits the document file as document data (such as a PDF file) to the image processing apparatus 1. When the image processing apparatus 1 receives the document data transmitted from the operation terminal 2, it executes various processes described later on the document data to extract the character string to be managed included in the form. For example, the image processing apparatus 1 extracts the classification (type) of each form, the date of each form, the amount (total amount, etc.), company information (issuer, destination, registration number, etc.), and the like. Further, the image processing apparatus 1 registers the extracted character string in a predetermined database. For example, every time the image processing apparatus 1 acquires document data of an invoice, it extracts a character string related to the content of the invoice (for example, issue date, invoice amount, issuer, etc.) from the document data and registers it in the database for managing invoices. Also, every time the image processing apparatus 1 acquires document data of a receipt, it extracts a character string related to the content of the receipt (for example, issue date, total amount, issuer, etc.) from the document data and registers it in the database for managing receipts. Thereby, each form can be stored and managed as electronic data. Further, the image processing apparatus 1 outputs the extracted character string to the operation terminal 2 or the like and presents the character recognition result to the user. Note that the image processing system 10 is not limited to form management and can be applied to various fields such as document search.

[0014] The image processing system 10 is an example of the information processing system of the present disclosure. Note that the information processing system of the present disclosure may be configured by the image processing apparatus 1 alone.

[0015] [Image Processing Apparatus 1] As shown in Figure 1, the image processing device 1 includes a control unit 11, a storage unit 12, an operation display unit 13, a communication unit 14, and the like. The image processing device 1 may consist of one or more cloud servers, or one or more physical servers.

[0016] The communication unit 14 is a communication interface for connecting the image processing device 1 to the network N1 by wire or wireless connection and for performing data communication with the operating terminal 2 via the network N1 in accordance with a predetermined communication protocol. The network N1 consists of, for example, the internet, a LAN, etc.

[0017] The operation display unit 13 is a user interface comprising a display unit such as a liquid crystal display or an organic EL display that displays various types of information, and an operation unit such as a mouse, keyboard, or touch panel that accepts input.

[0018] The storage unit 12 is a non-volatile storage unit such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory that stores various types of information. The storage unit 12 stores a control program that causes the control unit 11 to execute the character extraction process described later. For example, the control program is non-temporarily recorded on a computer-readable recording medium such as a CD or DVD, read by a reading device (not shown) such as a CD drive or DVD drive provided by the image processing device 1, and stored in the storage unit 12. The control program may also be distributed from a cloud server and stored in the storage unit 12.

[0019] Furthermore, the storage unit 12 stores document data (such as PDF files) such as forms obtained from the operating terminal 2.

[0020] Figure 2 shows an invoice as an example of a form (document data P1). As shown in Figure 2, the invoice contains strings such as the document type ("Invoice"), issue date, issuer's contact information (address, telephone number, fax number, person in charge), invoice amount, product name, quantity, standard price, discount amount, subtotal, consumption tax, and total amount. The user scans the invoice using the operation terminal 2 and uploads the document data P1 (PDF file) to the image processing device 1. The control unit 11 acquires the document data P1 of the invoice and stores it in the storage unit 12. In the document data P1 in Figure 2, code A1 represents a text object, code A2 represents an image object, and code A3 represents a path object.

[0021] In another embodiment, the control unit 11 may acquire the document file of the form created on the operation terminal 2 and store the document file in the storage unit 12.

[0022] The control unit 11 includes control devices such as a CPU, ROM, and RAM. The CPU is a processor that performs various arithmetic operations. The ROM stores control programs such as a BIOS and OS in advance to allow the CPU to perform various operations. The RAM stores various information and is used as a temporary storage memory (work area) for the various operations performed by the CPU. The control unit 11 controls the image processing device 1 by executing various control programs stored in advance in the ROM or storage unit 12 using the CPU.

[0023] As shown in Figure 1, the control unit 11 includes various processing units such as an acquisition processing unit 111, an extraction processing unit 112, a replacement processing unit 113, a deletion processing unit 114, a modification processing unit 115, a rendering processing unit 116, a recognition processing unit 117, and an output processing unit 118. The control unit 11 functions as one of these various processing units by executing various processes according to the control program. Some or all of the processing units included in the control unit 11 may be composed of electronic circuits. The control program may be a program that causes multiple processors to function as the various processing units.

[0024] [Character extraction process (overall flow)] Here, when the control unit 11 acquires document data P1 (document file), it performs rendering on the document data P1 and performs optical character recognition (OCR) on the image data converted into an image by the rendering process to extract the characters of the necessary items in the form. In addition, as a preprocessing step for rendering, the control unit 11 performs specific character replacement processing, which replaces characters that may be corrupted in the text objects in the document data P1 with characters that can be recognized by OCR processing; specific image deletion processing, which deletes the image object of the seal impression in the document data P1; and background background change processing, which changes the color of the text objects and background in the document data P1. Figure 3 shows an example of the procedure for the character extraction process including each of the above processes.

[0025] This disclosure can be understood as a character extraction method that performs one or more steps included in the character extraction process. Furthermore, one or more steps included in the character extraction process described herein may be omitted as appropriate. In addition, the execution order of each step in the character extraction process may differ to the extent that similar effects are produced. Furthermore, although this description uses the case in which the control unit 11 of the image processing device 1 performs each step in the character extraction process as an example, in other embodiments, one or more processors may distribute and execute each step in the character extraction process. In addition, when the control unit 11 acquires document data P1 from each of the multiple operation terminals 2, it is possible to execute the character extraction process in parallel for each piece of document data P1.

[0026] In step S1, the control unit 11 (acquisition processing unit 111) determines whether or not it has acquired the document data P1 (document file, PDF file) of the form to be processed from the operation terminal 2. If the control unit 11 acquires the document data P1 from the operation terminal 2 (S1:Yes), it proceeds to step S2. The control unit 11 waits until it acquires the document data P1 from the operation terminal 2 (S1:No).

[0027] In step S2, the control unit 11 (extraction processing unit 112) determines whether or not there is extractable embedded text information in the document data P1. If the control unit 11 determines that the document data P1 contains the text information (S2: Yes), it proceeds to step S3. On the other hand, if the control unit 11 determines that the document data P1 does not contain the text information (S2: No), it proceeds to step S5.

[0028] In step S3, the control unit 11 (extraction processing unit 112) extracts the embedded text information from the document data P1.

[0029] In step S4, the control unit 11 (extraction processing unit 112) extracts the content (example characters) of items required for the form from the extracted embedded text information. For example, the extraction processing unit 112 extracts items such as category, amount, date, customer, and registration number. The extraction processing unit 112 registers each extracted string into the database that manages the invoices.

[0030] In step S5, the control unit 11 (extraction processing unit 112) extracts page information from the document data P1. For example, the extraction processing unit 112 extracts the page's vertical and horizontal dimensions, number of objects, resolution, color information, etc.

[0031] Next, the control unit 11 performs the following steps S6 to S8 for each object included in the document data P1 other than the embedded text information. In step S6, the control unit 11 (replacement processing unit 113) performs the specific character replacement processing. Specifically, the control unit 11 replaces characters that are garbled in the text objects included in the document data P1 with characters that can be recognized in OCR processing (substitute characters). A specific example of the specific character replacement processing (see Figure 5) will be described later. The replacement processing unit 113 is an example of the correction processing unit of this disclosure.

[0032] In step S7, the control unit 11 (deletion processing unit 114) executes the specific image deletion process. Specifically, the control unit 11 deletes the image object of the seal impression included in the document data P1. A specific example of the specific image deletion process (see Figure 6) will be described later. The deletion processing unit 114 is an example of the correction processing unit of this disclosure.

[0033] In step S8, the control unit 11 (change processing unit 115) executes the background background change processing. Specifically, the control unit 11 changes the color of the text objects and background in the document data P1. A specific example of the background background change processing (see Figure 8) will be described later. The change processing unit 115 is an example of the correction processing unit of this disclosure.

[0034] The control unit 11 repeatedly executes the processes in steps S6 to S8 described above for the number of objects contained in the document data P1. The order of the specific character replacement process, the specific image deletion process, and the background background change process is not restricted; they may be performed in any order, and each process may be executed in parallel.

[0035] After the control unit 11 has performed the processes in steps S6 to S8 described above for all objects, it moves the process to step S9. In step S9, the control unit 11 (rendering processing unit 116) performs rendering processing. Specifically, the rendering processing unit 116 performs rendering processing (image processing) on ​​the document data P1 to generate image data for optical character recognition (OCR). The rendering processing unit 116 is an example of a generation processing unit in this disclosure.

[0036] In step S10, the control unit 11 (recognition processing unit 117) performs OCR processing on the image data to recognize characters. For example, the control unit 11 uses AI learning type OCR to recognize characters. The control unit 11 performs preprocessing such as character inversion and italicization before the OCR processing.

[0037] In step S11, the control unit 11 (output processing unit 118) outputs the character recognition results (OCR results). Specifically, the output processing unit 118 extracts the content (strings) of the items required for the form from the character recognition results. The extraction processing unit 112 registers each extracted string into the database that manages the invoices.

[0038] The following describes specific examples of the character replacement process (step S6), the specific image deletion process (step S7), and the background background modification process (step S8) that are included in the character extraction process.

[0039] [Specific character replacement process] In the aforementioned specific character replacement process, when the extraction processing unit 112 extracts a text object from the document data P1, the replacement processing unit 113 replaces the character to be corrected (to be replaced) with a predetermined character that has been registered in advance in association with the character to be corrected, if the string of the text object contains the character to be corrected. Specifically, the replacement processing unit 113 refers to a replacement character list D1 (see Figure 4), which stores the name of the font that is corrupted, the character that is corrupted, and the character to be replaced (replacement character) in association with each other, identifies the replacement character associated with the character to be corrected that is corrupted, and replaces that character with the replacement character.

[0040] Figure 4 shows an example of the replacement character list D1. The replacement character list D1 is pre-registered with the association between font names that may cause garbled text, characters that may cause garbled text, and alternative characters to replace those characters. Note that characters that may cause garbled text are not limited to text characters, but may also include emojis, symbols, etc.

[0041] Figure 5 shows an example of the procedure for the specific character replacement process.

[0042] <Step S61> In step S61, the control unit 11 (extraction processing unit 112) extracts object information from the document data P1. Specifically, the control unit 11 extracts text objects, drawing objects (image objects, path objects), etc. from the document data P1 (see Figure 2).

[0043] <Step S62> In step S62, the control unit 11 (replacement processing unit 113) determines whether the extracted target object is a text object. If the target object is a text object (S62:Yes), the replacement processing unit 113 proceeds to step S63. On the other hand, if the target object is not a text object (S62:No), the replacement processing unit 113 proceeds to step S7 (specific image deletion processing).

[0044] <Step S63> In step S63, the control unit 11 (replacement processing unit 113) obtains font information to be used in the rendering process from the text object.

[0045] <Step S64> In step S64, the control unit 11 (replacement processing unit 113) determines whether the acquired font is a font to be replaced. Specifically, the replacement processing unit 113 determines whether the acquired font is a font that may cause garbled characters. The replacement processing unit 113 determines that the acquired font is a font to be replaced if it matches the "garbled font name" in the replacement character list D1 (see Figure 4). If the replacement processing unit 113 determines that the acquired font is a font to be replaced (S64:Yes), it proceeds to step S65. On the other hand, if the replacement processing unit 113 determines that the acquired font is not a font to be replaced (S64:No), it proceeds to step S7 (specific image deletion processing).

[0046] <Step S65> In step S65, the control unit 11 (replacement processing unit 113) searches for the character to be corrupted within the string of the text object. Specifically, the replacement processing unit 113 searches the string of the extracted text object for a character that matches the "garbled character" registered in the replacement character list D1.

[0047] <Step S66> In step S66, the control unit 11 (replacement processing unit 113) proceeds to step S67 if the text string of the text object contains characters that will be corrupted (S66: Yes), and proceeds to step S7 (specific image deletion processing) if there are no characters that will be corrupted (S66: No).

[0048] <Step S67> In step S67, the control unit 11 (replacement processing unit 113) replaces the garbled characters with pre-registered characters. Specifically, the replacement processing unit 113 replaces the target characters with "alternative characters" registered in the replacement character list D1. Once the replacement processing unit 113 has finished replacing the garbled characters with other characters (alternative characters), it moves the process to step S7 (specific image deletion process).

[0049] As described above, the replacement processing unit 113 executes the specific character replacement process. For example, the character "￥" in a particular font may be corrupted into "\" during the rendering process and misrecognized as "1" during the OCR process. In this case, the replacement processing unit 113 replaces the character "￥" with the character "Y", which is less likely to be corrupted during the rendering process. As a result, the rendering processing unit 116 can create an image of the character "Y" without it being corrupted. Then, the recognition processing unit 117 can recognize the character "Y" as "Y" in the OCR process (step S10 in Figure 3) without misrecognizing it as "1". In addition, the output processing unit 118 can extract "Y + number" as an amount in the form required item extraction process (step S11 in Figure 3).

[0050] As described above, in the specific character replacement process, the control unit 11 registers information about the font name that will cause garbled characters, the character information that will cause garbled characters, and the character information to be replaced in advance in a database (replacement character list D1 (see Figure 5)). Also, if a text object exists, the control unit 11 determines whether the font used to render that text object is one of the fonts that will cause garbled characters that has been registered in advance. If a font that will cause garbled characters is used, the control unit 11 checks whether there are any garbled characters in it, and if there are, replaces them with predetermined characters that have been registered in advance. As a result, the control unit 11 can extract the correct amount, for example, when extracting an amount from OCR results, by setting conditions such as treating a string of numbers starting with "Y" as "￥" and ignoring characters other than numbers.

[0051] [Specific image deletion process] In the aforementioned specific image deletion process, when the extraction processing unit 112 extracts an image object, which is a drawing object, from the document data P1, the deletion processing unit 114 determines whether the image object is a seal impression image to be deleted based on the number of pixels or area of ​​pixels of a predetermined color among the multiple pixels constituting the image object, and deletes the image object if it is determined to be a seal impression image. Furthermore, the deletion processing unit 114 determines whether the image object is a seal impression image based on the ratio of the number of red pixels to the number of pixels of colors other than red among the multiple pixels constituting the image object, or based on the aspect ratio of the rectangular size of the area of ​​red pixels.

[0052] For example, if the color format of the image object is 24-bit or 32-bit, the deletion processing unit 114 counts the number of red, black, and white pixels and determines whether the image object is the seal impression image based on the ratio of each pixel number or the aspect ratio of the red pixels. If the color format of the image object is 8-bit, the deletion processing unit 114 counts the number of black, gray, and white pixels, and if gray pixels are present, it treats gray as red and recounts, and if no gray pixels are present, it treats white as red and black as white and recounts, and determines whether the image object is the seal impression image based on the ratio of each pixel number or the aspect ratio of the red pixels. Furthermore, if the color format of the image object is 1 bit, the deletion processing unit 114 counts the number of black and white pixels, then counts again treating white as red and black as white, and determines whether the image object is the seal image based on the ratio of each pixel count or the aspect ratio of the red pixels.

[0053] Figure 6 shows an example of the procedure for the specific image deletion process.

[0054] <Step S71> In step S71, the control unit 11 (extraction processing unit 112) extracts object information from the document data P1. Specifically, the control unit 11 extracts text objects, drawing objects (image objects, path objects), etc., from the document data P1.

[0055] <Step S72> In step S72, the control unit 11 (deletion processing unit 114) determines whether the extracted target object is an image object. If the target object is an image object (S72:Yes), the deletion processing unit 114 proceeds to step S73. On the other hand, if the target object is not an image object (S72:No), the deletion processing unit 114 proceeds to step S8 (background background change processing).

[0056] <Step S73> In step S73, the control unit 11 (deletion processing unit 114) obtains image information to be used in the rendering process from the image object. Specifically, the deletion processing unit 114 obtains the image size, the number of bits per pixel (color format), etc. from the image object. If the image size is too small (for example, less than 24 pixels vertically and horizontally, 72 dpi or less), or if the width of the image is 90% or more of the width of the page size and the height is less than 50% of the page size, the deletion processing unit 114 excludes the image from the specific image deletion process and proceeds to step S8 (background background change processing).

[0057] <Step S74> In step S74, the control unit 11 (deletion processing unit 114) determines whether the color format of the image object is 24-bit (RGB) or 32-bit (RGBA). If the color format of the image object is 24-bit or 32-bit (S74: Yes), the deletion processing unit 114 proceeds to step S75. On the other hand, if the color format of the image object is not 24-bit or 32-bit (S74: No), the deletion processing unit 114 proceeds to step S741.

[0058] <Step S75> In step S75, the control unit 11 (deletion processing unit 114) counts the number of red, black, and white pixels. Once the deletion processing unit 114 has counted the number of pixels of each color, it proceeds to step S76.

[0059] <Step S741> In step S741, the control unit 11 (deletion processing unit 114) determines whether the color format of the image object is 8-bit (Index Color). If the color format of the image object is 8-bit (S741: Yes), the deletion processing unit 114 proceeds to step S742. On the other hand, if the color format of the image object is not 8-bit (S741: No), the deletion processing unit 114 proceeds to step S744.

[0060] <Step S742> In step S742, the control unit 11 (deletion processing unit 114) counts the number of black, gray, and white pixels.

[0061] <Step S743> In step S743, the control unit 11 (deletion processing unit 114) recounts the number of pixels for each color, treating gray pixels as red pixels if gray pixels exist. After counting the number of pixels for each color, the deletion processing unit 114 proceeds to step S76.

[0062] <Step S744> In step S744, the control unit 11 (deletion processing unit 114) counts the number of black and white pixels.

[0063] <Step S745> In step S745, the control unit 11 (deletion processing unit 114) treats white as red and black as white and recounts the number of pixels for each color. Once the deletion processing unit 114 has counted the number of pixels for each color, it proceeds to step S76.

[0064] <Step S76> In step S76, the control unit 11 (deletion processing unit 114) calculates the proportion of each pixel count and the aspect ratio of the red pixel area. Specifically, the deletion processing unit 114 calculates the composition ratio of the number of red pixels and the aspect ratio of the rectangular size of the red pixel area.

[0065] <Step S77> In step S77, the control unit 11 (deletion processing unit 114) determines whether the image object is an image to be deleted (a seal impression image) based on the composition ratio of red pixels and the aspect ratio of the red pixel area. If the deletion processing unit 114 determines that the image object is a seal impression image (S77: Yes), it proceeds to step S78. On the other hand, if the deletion processing unit 114 determines that the image object is not a seal impression image (S77: No), it proceeds to step S8 (background background change processing).

[0066] Furthermore, the deletion processing unit 114 excludes images from the specific image deletion process if there are too many red pixels (when the size of the image object is 95% or more of the page size and the number of red pixels is 4% or more of the size of the image object) or if black pixels are included (when the size of the image object is 95% or more of the page size and the number of black pixels is 0.02% or more of the size of the image object).

[0067] Furthermore, the deletion processing unit 114 determines an image object to be a seal image if, for example, (1) the area where red pixels exist is less than half of the page size in both the vertical and horizontal directions, and the number of red pixels counted is greater than 7 / 50 (14%) of the total number of pixels in the image object, and (2) the area where red pixels exist is less than half of the page size in both the vertical and horizontal directions, red pixels exist, the total count of red and white pixels is greater than 9 / 10 (90%) of the total number of pixels in the image object, and the count of black pixels is less than 5% of the total number of pixels in the image object. However, the deletion processing unit 114 excludes images from the specific image deletion process if black pixels make up more than 1% of the total number of pixels in the image object, the vertical center of the area where black pixels exist is within the area where red pixels exist, and the horizontal area where black pixels exist overlaps with the area where red pixels exist in any way.

[0068] Furthermore, the deletion processing unit 114 calculates the aspect ratio of the rectangular size of the red pixel area and makes a final determination to delete it based on the following three conditions. Specifically, the deletion processing unit 114 will delete an image object if (1) it is determined to be a red seal image for an image object with 24 bits or more, and the value obtained by subtracting the aspect ratio of the rectangular size from 1 (square) is less than 0.4 (first condition), or (2) it is determined to be a red seal image for an image object with 8 bits or less, and the value obtained by subtracting the aspect ratio of the rectangular size from 1 (square) is less than 0.296 (second condition), or (3) it is a red pixel area located approximately inside an image object that has already been deleted (third condition). In this way, the deletion processing unit 114 may delete an image object if any of the above first to third conditions are met. However, if the deletion processing unit 114 has already deleted an image object that is a full-color image (an image object with 24 bits or more), and the image object being judged is not a full-color image (8 bits or less), it will change the image object to be excluded from deletion even if it would otherwise be subject to deletion under the above conditions.

[0069] <Step S78> In step S78, the control unit 11 (deletion processing unit 114) deletes the image object that it has determined to be a seal impression image. Once the deletion processing unit 114 has finished deleting the seal impression images to be deleted, it proceeds to step S8 (background background change processing).

[0070] As described above, the deletion processing unit 114 executes the specific image deletion process. For example, as shown in Figure 7A, if an image object P11 includes a seal impression image A2 within the page size, the deletion processing unit 114 determines that the image object P11 is a target for deletion and deletes it. Also, as shown in Figure 7B, if an image object P12 consists only of the seal impression image A2, the deletion processing unit 114 determines that the image object P12 is a target for deletion and deletes it. Also, as shown in Figure 7C, if the seal impression image consists of multiple images A21 and A22, the deletion processing unit 114 determines that an image object P13 containing image A21 and an image object P14 containing image A22 are targets for deletion and deletes them. Note that, as shown in Figure 7D, if the image contains both the seal impression image and the company name, the deletion processing unit 114 excludes the image object P15 from the deletion targets.

[0071] [Background background change process] In the background background modification process, when the extraction processing unit 112 extracts a text object from the document data P1, the modification processing unit 115 changes the color of at least one of the stroke and the fill based on the color information of the stroke, which is the outline portion of the characters in the text object, and the color information of the fill portion, which is the filled portion within the area of ​​the characters. Also, when the extraction processing unit 112 extracts a path object, which is a drawing object, from the document data P1, the modification processing unit 115 changes the color of at least one of the stroke and the fill based on the rectangle size information of the path object, the stroke color information, and the fill color information.

[0072] Figure 8 shows an example of the procedure for the background substrate modification process.

[0073] <Step S81> In step S81, the control unit 11 (extraction processing unit 112) extracts object information from the document data P1. Specifically, the control unit 11 extracts text objects, drawing objects (image objects, path objects), etc., from the document data P1.

[0074] <Step S82> In step S82, the control unit 11 (modification processing unit 115) determines whether the extracted target object is a text object. If the target object is a text object (S82:Yes), the modification processing unit 115 proceeds to step S83. On the other hand, if the target object is not a text object (S82:No), the modification processing unit 115 proceeds to step S86.

[0075] <Step S83> In step S83, the control unit 11 (modification processing unit 115) obtains color information from the text object to be used in the rendering process. Specifically, the modification processing unit 115 obtains information on the stroke color (color of the outline of the characters) and the fill color (color of the fill area of ​​the characters).

[0076] <Step S84> In step S84, the control unit 11 (change processing unit 115) determines whether the acquired color is the color to be changed. If the change processing unit 115 determines that the acquired color is the color to be changed (S84: Yes), it proceeds to step S85. On the other hand, if the change processing unit 115 determines that the acquired color is not the color to be changed (S84: No), it proceeds to step S86. For example, the change processing unit 115 determines that the color to be changed is white if the stroke color and / or the fill color is white.

[0077] <Step S85> In step S85, the control unit 11 (change processing unit 115) changes the color of the character to be changed. Specifically, the change processing unit 115 changes the white stroke color to black and the white fill color to black. Once the change processing unit 115 has finished changing the color of the character to be changed, it moves the process to step S9 (rendering process).

[0078] <Step S86> In step S86, the control unit 11 (modification processing unit 115) determines whether the extracted target object is a path object. If the target object is a path object (S86:Yes), the modification processing unit 115 proceeds to step S87. On the other hand, if the target object is not a path object (S86:No), the modification processing unit 115 proceeds to step S9 (rendering process).

[0079] <Step S87> In step S87, the control unit 11 (modification processing unit 115) obtains color information from the text object. Specifically, the modification processing unit 115 obtains information on the stroke color (line color) and the fill color (color of the area enclosed by the line).

[0080] <Step S88> In step S88, the control unit 11 (change processing unit 115) determines whether the acquired color is the color to be changed. If the change processing unit 115 determines that the acquired color is the color to be changed (S88: Yes), it proceeds to step S89. On the other hand, if the change processing unit 115 determines that the acquired color is not the color to be changed (S88: No), it proceeds to step S9. For example, the change processing unit 115 determines that the colors to be changed are white when the stroke color is white and the fill color is black.

[0081] <Step S89> In step S89, the control unit 11 (change processing unit 115) changes the background color of the subject to be changed. Specifically, the change processing unit 115 changes the black fill color to white or gray. Once the change processing unit 115 has finished changing the background color of the subject to be changed, it moves the process to step S9 (rendering process).

[0082] As described above, the modification processing unit 115 executes the background background modification process. Note that the modification processing unit 115 excludes small rectangles where neither the stroke color nor the fill color is white. Also, the modification processing unit 115 excludes path objects that represent thin horizontal lines.

[0083] Furthermore, the modification processing unit 115 determines whether an area is subject to deletion if the stroke color is white and the fill color is black. If it determines that it is not subject to deletion, it changes only the stroke color to black. This is to allow for later determination of the condition when both the stroke color and fill color are black.

[0084] Furthermore, the modification processing unit 115 determines whether the fill color is gray or a colored color in the following cases: (1) when the fill color is other than white and both the stroke color and fill color are other than black, (2) when the stroke color is black and the fill color is other than black, (3) when the stroke color is white and the fill color is other than black, or (4) when the stroke color is a dark color and the fill color is a light color. After determining whether it is an area to be deleted, it changes the fill color to light gray.

[0085] Furthermore, the modification processing unit 115 changes the fill color from white to gray because in some cases, the cells of the table are represented by a rectangle filled with a color rather than a line. For example, in the table shown in Figure 9A, the rectangles for "Invoice Amount" and "Bank Transfer" are represented only by the fill color. In this case, if the fill color is changed to white, the table cannot be recognized in OCR preprocessing, as shown in Figure 9B. Therefore, in this embodiment, as shown in Figure 9C, the modification processing unit 115 changes the fill color to gray. This makes it possible to recognize the table in OCR preprocessing.

[0086] The modification processing unit 115 may determine whether or not the area is subject to deletion by the following method.

[0087] For example, the modification processing unit 115 determines an area to be deleted if any of the following conditions are met: 1) Bézier curves are not used to draw the path object, the length of the straight line set for drawing is greater than or equal to a certain length (the length of the vertical and horizontal lines must be different), the path object represents a rectangle, and its size is greater than or equal to a certain size; 2) Bézier curves are not used to draw the path object, the fill color is gray or a colored path object, the path object represents a rectangle, and its size is greater than or equal to a certain size; 3) Bézier curves are not used to draw the path object, the fill color is not gray or a colored path object, and the vertical line is greater than or equal to a certain size.

[0088] Furthermore, the modification processing unit 115 may count the following two conditions (the fourth condition and the fifth condition), and if the total count is 4 or less and the size of the path object is greater than or equal to a certain size, it may determine that the area is subject to deletion. The fourth condition counts when a Bézier curve is used to draw the path object on a path segment basis within the path object. The fifth condition counts when the path object is a horizontal line of a certain size or larger in length, rather than a path object with a fill color of gray or color. However, even if an area is determined to be subject to deletion by satisfying any of the above three conditions (the first to third conditions) or the count or other conditions based on the above two conditions (the fourth condition and the fifth condition), it will not be subject to deletion if the rectangle size is greater than or equal to a certain size.

[0089] Figure 10A shows an example of a path object with a white line filled with black. The path object shown in Figure 10A is not a text object, so the inside of the characters cannot be filled. Therefore, simply changing the fill color to white will not allow the text "Invoice" to be recognized, as shown in Figure 10B. In this embodiment, the modification processing unit 115 changes the fill color to white and the stroke color to black, as shown in Figure 10C. This makes it possible to properly recognize inverted characters (white outline characters) in OCR processing.

[0090] As described above, the image processing device 1 according to this embodiment extracts at least one of the target objects (text objects and drawing objects, image objects and path objects) from document data P1. If the extracted target object matches a pre-registered target object, it performs a predetermined correction process (specific character replacement process, specific image deletion process, and background background modification process) on the target object, and performs a rendering process on the document data P1 with the corrected target object to generate image data for character recognition. The image processing device 1 also performs character recognition processing on the generated image data and outputs the recognition result.

[0091] According to the above configuration, for example, garbled characters contained in a text object can be replaced with other characters before rendering, thereby enabling proper rendering of the text object. Furthermore, if a seal image overlaps a text object, the seal image can be removed before rendering the text object. Additionally, the background color and text color can be changed during rendering. This allows for proper rendering of document data P1, thereby improving the accuracy of character recognition in OCR processing.

[0092] The information processing system disclosed herein may be a system that performs any one of the following processes: the specific character replacement process, the specific image deletion process, and the background background modification process, or it may be a system that performs a combination of multiple processes. Furthermore, in the information processing system disclosed herein, the order in which the specific character replacement process, the specific image deletion process, and the background background modification process are performed is not limited and may be performed in any order.

[0093] The control unit 11 of the image processing device 1 controls the entire image processing device 1. The control unit 11 realizes various functions by reading and executing various programs stored in the storage unit 12 (for example, storage or ROM). The control unit 11 may be realized by one or more control devices / arithmetic units (CPU (Central Processing Unit), SoC (System on a Chip)). The control unit 11 may also be composed of one or more control circuits (electronic circuits).

[0094] [Note on disclosure] The following is an overview of the disclosures extracted from the above-described embodiments. Note that each configuration and processing function described below can be selected and combined as desired.

[0095] <Note 1> An extraction processing unit that extracts at least one of the target objects, text objects and drawing objects, from document data, A correction processing unit executes a predetermined correction process on the target object when the target object extracted by the extraction processing unit matches a pre-registered object to be processed. A generation processing unit that performs rendering processing on the document data from which the target object has been corrected by the correction processing unit to generate image data for character recognition, An information processing system equipped with the following features.

[0096] <Note 2> When the extraction processing unit extracts an image object, which is a drawing object, from the document data, The correction processing unit determines whether the image object is a seal impression image to be deleted based on the number of pixels or area of ​​pixels of a predetermined color among the multiple pixels constituting the image object, and deletes the image object if it is determined to be a seal impression image. The information processing system described in Appendix 1.

[0097] <Note 3> The correction processing unit determines whether the image object is the seal image based on the ratio of the number of red pixels to the number of pixels of colors other than red among the plurality of pixels constituting the image object, or the aspect ratio of the rectangular size of the red pixel region. The information processing system described in Appendix 2.

[0098] <Note 4> The correction processing unit, If the color format of the image object is 24-bit or 32-bit, the number of red, black, and white pixels is counted, and it is determined whether the image object is the seal image based on the ratio of each pixel number or the aspect ratio of the red pixels. If the color format of the image object is 8 bits, the number of black, gray, and white pixels is counted, and if gray pixels are present, the gray pixels are treated as red and the count is repeated. If no gray pixels are present, white pixels are treated as red and black pixels are treated as white and the count is repeated. Based on the ratio of each pixel count or the aspect ratio of the red pixels, it is determined whether the image object is the seal image. If the color format of the image object is 1 bit, the number of black and white pixels is counted, and then the white pixels are treated as red and the black pixels as white and counted again, and it is determined whether the image object is the seal image based on the ratio of each pixel number or the aspect ratio of the red pixels. The information processing system described in Appendix 3.

[0099] <Note 5> When the extraction processing unit extracts the text object from the document data, The correction processing unit, when the string of the text object contains a character to be corrected, replaces that character with a predetermined character that has been registered in advance and associated with the character to be corrected. An information processing system as described in any of the appendices 1 to 4.

[0100] <Note 6> The correction processing unit refers to a storage unit that stores the name of the font that is garbled, the garbled character, and the substitute character to be replaced in association with each other, identifies the substitute character associated with the garbled character to be corrected, and replaces the character with the substitute character. The information processing system described in Appendix 5.

[0101] <Note 7> When the extraction processing unit extracts the text object from the document data, The correction processing unit changes the color of at least one of the stroke and the fill based on the color information of the stroke, which is the outline portion of the characters in the text object, and the color information of the fill portion, which is the filled portion within the area of ​​the characters. An information processing system as described in any of the appendices 1 to 6.

[0102] <Note 8> When the extraction processing unit extracts a path object, which is a drawing object, from the document data, The correction processing unit changes the color of at least one of the stroke and the fill based on the information of the rectangular size of the path object, the color information of the stroke which is the line portion of the path object, and the color information of the fill portion which is the filled area within the region enclosed by the line. An information processing system as described in any of the appendices 1 to 7.

[0103] <Note 9> A recognition processing unit that performs character recognition processing on the image data generated by the generation processing unit, An output processing unit that outputs the recognition result from the aforementioned recognition processing unit, An information processing system described in any of the appendices 1 to 8, further comprising the above.

[0104] <Note 10> Extracting at least one of the target objects, either text objects or drawing objects, from document data, If the aforementioned target object matches a pre-registered object to be processed, a predetermined correction process is performed on the target object. The process involves performing a rendering process on the corrected document data of the target object to generate image data for character recognition, An information processing method performed by one or more processors.

[0105] <Note 11> Extracting at least one of the target objects, either text objects or drawing objects, from document data, If the aforementioned target object matches a pre-registered object to be processed, a predetermined correction process is performed on the target object. The process involves performing a rendering process on the corrected document data of the target object to generate image data for character recognition, An information processing program for causing one or more processors to execute, or a non-temporary computer-readable recording medium on which such information processing program is recorded. [Explanation of Symbols]

[0106] 1: Image processing device 2: Operating terminal 10: Information Processing Systems 11: Control Unit 12: Storage section 111: Acquisition Processing Unit 112: Extraction Processing Unit 113: Replacement Processing Unit 114: Deletion Processing Unit 115: Change Processing Unit 116: Rendering Processing Unit 117: Recognition Processing Unit 118: Output Processing Unit D1: List of replacement characters

Claims

1. An extraction processing unit that extracts at least one of the target objects, text objects and drawing objects, from document data, If the target object extracted by the extraction processing unit matches a pre-registered object to be processed, the correction processing unit executes a predetermined correction process on the target object. A generation processing unit that performs rendering processing on the document data from which the target object has been corrected by the correction processing unit to generate image data for character recognition, An information processing system equipped with the following features.

2. When the extraction processing unit extracts an image object, which is a drawing object, from the document data, The correction processing unit determines whether the image object is a seal impression image to be deleted based on the number of pixels or area of ​​pixels of a predetermined color among the multiple pixels constituting the image object, and deletes the image object if it is determined to be a seal impression image. The information processing system according to claim 1.

3. The correction processing unit determines whether the image object is the seal image based on the ratio of the number of red pixels to the number of pixels of colors other than red among the multiple pixels constituting the image object, or the aspect ratio of the rectangular size of the red pixel region. The information processing system according to claim 2.

4. The correction processing unit, If the color format of the image object is 24-bit or 32-bit, the number of red, black, and white pixels is counted, and it is determined whether the image object is the seal image based on the ratio of each pixel number or the aspect ratio of the red pixels. If the color format of the image object is 8 bits, the number of black, gray, and white pixels is counted, and if gray pixels are present, the gray pixels are treated as red and the count is restarted. If no gray pixels are present, white pixels are treated as red and black pixels are treated as white and the count is restarted. Based on the ratio of each pixel count or the aspect ratio of the red pixels, it is determined whether the image object is the seal image. If the color format of the image object is 1 bit, the number of black and white pixels is counted, and then the white pixels are treated as red and the black pixels as white and counted again, and it is determined whether the image object is the seal image based on the ratio of each pixel number or the aspect ratio of the red pixels. The information processing system according to claim 3.

5. When the extraction processing unit extracts the text object from the document data, The correction processing unit, when the string of the text object contains a character to be corrected, replaces that character with a predetermined character that has been registered in advance and associated with the character to be corrected. The information processing system according to claim 1.

6. The correction processing unit refers to a storage unit that stores the name of the font that is garbled, the garbled character, and the substitute character to be replaced in association with each other, identifies the substitute character associated with the garbled character to be corrected, and replaces the character with the substitute character. The information processing system according to claim 5.

7. When the extraction processing unit extracts the text object from the document data, The correction processing unit changes the color of at least one of the stroke and the fill based on the color information of the stroke, which is the outline portion of the characters in the text object, and the color information of the fill, which is the filled portion within the area of ​​the characters. The information processing system according to claim 1.

8. When the extraction processing unit extracts a path object, which is a drawing object, from the document data, The correction processing unit changes the color of at least one of the stroke and the fill based on the information of the rectangular size of the path object, the color information of the stroke which is the line portion of the path object, and the color information of the fill portion which is the filled area within the region enclosed by the line. The information processing system according to claim 1.

9. A recognition processing unit that performs character recognition processing on the image data generated by the generation processing unit, An output processing unit that outputs the recognition result from the aforementioned recognition processing unit, The information processing system according to any one of claims 1 to 8, further comprising:

10. Extracting at least one of the target objects, either text objects or drawing objects, from document data, If the aforementioned target object matches a pre-registered object to be processed, a predetermined correction process is performed on the target object. The process involves performing a rendering process on the corrected document data of the target object to generate image data for character recognition, An information processing method performed by one or more processors.

11. Extracting at least one of the target objects, either text objects or drawing objects, from document data, If the aforementioned target object matches a pre-registered object to be processed, a predetermined correction process is performed on the target object. The process involves performing a rendering process on the corrected document data of the target object to generate image data for character recognition, An information processing program that causes one or more processors to execute.

Images