Image processing apparatus, image processing method, and program

Abstract

To standardize the font size corresponding to character images in cells which are not adjacent to each other in a table printed on a recording medium.SOLUTION: An image processing apparatus includes: a font size temporary determination unit 141 which temporarily determines a font size of a cell of interest, on the basis of a character image in a cell adjacent to the cell of interest; a cell attribute detection unit 143 which detects an attribute representing characteristics of a character image included in the cell of interest; a font size group classification unit 144 which classifies character images in all cells included in image data, on the basis of the attribute of the character image in the cell and the font size temporarily determined for the character image in the cell; and a font size final determination unit 145 which determines a font size for each of results of the classification, on the basis of the results classified by the font size group classification unit 144 and the font size temporarily determined.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, an image processing method, and a program.

Background Art

[0002] Conventionally, by returning a document printed on a recording medium to the electronic document data before printing, the electronic document data can be reused, improving convenience. Therefore, a technology (Office conversion, Word conversion) is known in which a document printed using a scanner is scanned, and after performing image area separation processing and the like, OCR (character recognition) is performed on the character image, and then converted into, for example, the Office Open XML Document format adopted by Microsoft Office (registered trademark).

[0003] Patent Document 1 aims to accurately detect the font size of a character image from image data without providing a table for obtaining the font size for each character code. For each character image in the image data, an outer circumscribing rectangle is detected, and the image data is divided into a plurality of blocks and set as a provisional determination area. A frequency distribution regarding the height of the outer circumscribing rectangle of each individual character image, which is a constituent element, is created for each provisional determination area unit, and a provisional determination area having two or more types of effective outer circumscribing rectangle heights is regarded as a normal determination area. For a provisional determination area not regarded as a normal determination area, it is integrated with other adjacent determination areas until the conditions of the normal determination area are satisfied, and for each normal determination area, the font size is determined using the maximum height among the outer circumscribing rectangles of the character images therein as a representative value. As described above, in Patent Document 1, an outer circumscribing rectangle is obtained for each character, the font size is detected based on a predetermined criterion, and the font size is determined using the maximum height among the outer circumscribing rectangles of the character images as a representative value. For this reason, in Patent Document 1, there is a problem that, for example, in a table printed on a recording medium by Excel, Word, or the like, the font sizes of non-adjacent areas (cells) cannot be unified.

[0004] In conventional character recognition processing, in the font size calculation process, characters in cells adjacent to a cell of interest and words were used as areas (cells) for determining the font size from the image data of the document. However, in a document printed with table cells such as Excel (registered trademark) or Word (registered trademark), there was a problem that the font sizes could not be unified for cells that were not adjacent to the cell of interest, that is, cells that were a certain distance away from the cell of interest.

Summary of the Invention

Problems to be Solved by the Invention

[0005] One embodiment of the present invention has been made in view of the above, and its object is to unify the font sizes corresponding to character images in cells that are not adjacent to each other in a table printed on a recording medium.

[0006] In order to solve the above problems, the invention according to claim 1 is an image processing apparatus that specifies a character code corresponding to a character image in each of a plurality of cells from image data related to a document including character images in the plurality of cells, and determines a font size corresponding to the character code. The image processing apparatus includes: a font size provisional determination unit that provisionally determines the font size of a cell of interest based on character images in cells adjacent to the cell of interest; a cell attribute detection unit that detects an attribute representing a feature of the character image included in the cell of interest; a font size group classification unit that classifies character images in all cells included in the image data based on the attribute of the character image in the cell and the font size provisionally determined for the character image in the cell; and a font size determination unit that determines the font size for each classification result based on the classification result by the font size group classification unit and the provisionally determined font size. This It is characterized by comprising.

Effects of the Invention

[0007] According to the present invention, in a table printed on a recording medium, it is possible to unify the font sizes corresponding to character images in cells that are not adjacent to each other.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0009] Hereinafter, the present invention will be described in detail with reference to the embodiments shown in the drawings. The present invention has the following configuration in order to unify the font sizes corresponding to character images in cells that are not adjacent to each other. That is, the image processing apparatus of the present invention is an image processing apparatus that identifies character codes corresponding to character images in each of a plurality of cells from image data related to a document containing character images in the plurality of cells, and determines a font size corresponding to the character codes, and includes a font size provisional determination unit that provisionally determines the font size of a cell of interest based on character images in cells adjacent to the cell of interest, a cell attribute detection unit that detects an attribute representing the characteristics of the character image included in the cell of interest, a font size group classification unit that classifies the character images in all cells included in the image data based on the attributes of the character images in the cell and the font size provisionally determined for the character image in the cell, and a font size determination unit that determines the font size for each classification result based on the classification result by the font size group classification unit and the font size provisionally determined. By having the above configuration, it is possible to unify the font sizes corresponding to character images in cells that are not adjacent to each other. The features of the present invention described above will be explained in detail with reference to the following drawings. However, the components, types, combinations, shapes, relative arrangements, etc. described in this embodiment are merely illustrative examples and not intended to limit the scope of the present invention only to them without specific description. Regarding the features of the present invention described above, it will be described in detail below with reference to the drawings.

[0010] <System Configuration> FIG. 1 is a diagram illustrating the configuration of a system including an image processing apparatus according to an embodiment of the present invention. System 1 includes an image forming apparatus 10 and two personal computers, PC80 and PC90, and is configured such that they transmit and receive data to and from each other via a communication network N. The image forming apparatus 10 is an MFP (Multifunction Peripheral) equipped with a scanner unit 31 that reads a document sheet, a printer unit 32 that forms an image on a sheet, and a communication function. When an ADF (Auto Document Feeder) is provided, the scanner unit 31 reads the document sheet set on the ADF. By using the scanner unit 31 and the printer unit 32, the image forming apparatus 10 provides users with functions of copy, printer, scanner, and facsimile. Also, by receiving instructions from a user via the operation panel 40 or receiving instructions from the PCs 80 and 90 via the communication network N, the image forming apparatus 10 executes these functions.

[0011] The image forming apparatus 10 also has a function of extracting text of electronic data from a scanned image read by the scanner unit 31, and a function of creating an editable document (e.g., a Microsoft Word document) based on the text and the scanned image, using optical character recognition technology (hereinafter, simply referred to as "character recognition" or "OCR" as necessary). Note that the PCs 80 and 90 are terminals (computers) used by general users who use the image forming apparatus 10. The image processing apparatus according to the present invention is composed of the PCs 80 and 90 shown in FIG. 1.

[0012] <Hardware Configuration of Image Forming Apparatus> FIG. 2 is a diagram showing an example of the hardware configuration of the image forming apparatus 10 shown in FIG. 1. As shown in FIG. 2, the image forming apparatus 10 includes a controller 20, a short-range communication circuit 60, an engine control unit 30, an operation panel 40, and a network I / F (Interface) 50. The controller 20 includes a CPU 25a which is the main component of the computer, a system memory (MEM-P) 22, a north bridge (NB) 25b, a south bridge (SB) 24, an ASIC (Application Specific Integrated Circuit) 26, a local memory (MEM-C) 27 which is a storage unit, an HDD (Hard Disk Drive) controller 28, and an HD 29 which is a storage unit. Also, the NB 25b and the CPU 25a are integrated by a SoC (System on a Chip) 25. The CPU 25a is a control unit that performs overall control of the image forming apparatus 10. The NB 25b is a bridge for connecting the CPU 25a, the MEM-P 22, and the SB 24, and has a memory controller that controls reading and writing to the MEM-P 22, a PCI (Peripheral Component Interconnect) master, and an AGP target.

[0013] The MEM-P 22 consists of a ROM 22a which is a memory for storing programs and data for realizing each function of the controller 20, and a RAM 22b which is used as a memory for expanding programs and data and for drawing during memory printing. Incidentally, the programs stored in the RAM 22b may be provided by being recorded on a computer-readable recording medium such as a CD-ROM, CD-R, or DVD in an installable or executable file format.

[0014] SB24 is a bridge for connecting NB25b with a PCI device and a peripheral device. ASIC26 is an IC (Integrated Circuit) for image processing applications having hardware elements for image processing, and has the role of a bridge for connecting the PCI bus 23, the HDD controller 28, and the MEM-C27 respectively. This ASIC26 consists of a PCI target and an AGP master, an arbiter (ARB) forming the core of the ASIC26, a memory controller for controlling the MEM-C27, a plurality of DMACs (Direct Memory Access Controllers) for performing operations such as rotation of image data by means of hardware logic, etc., and a PCI unit for performing data transfer via the PCI bus 23 between the scanner unit 31 and the printer unit 32. Incidentally, interfaces for USB (Universal Serial Bus) and IEEE1394 (Institute of Electrical and Electronics Engineers 1394) may be connected to the ASIC26.

[0015] MEM-C27 is a local memory used as a copy image buffer and a code buffer. HD29 is a storage that pre-stores programs for executing operations described later and various control data, and is also used for applications such as accumulation of image data, and accumulation of font data and forms used at the time of printing. HD29 controls reading or writing of data to HD29 according to the control of the CPU25a. In addition, the short-range communication circuit 60 is provided with a communication circuit 60a such as NFC and Bluetooth (registered trademark), and realizes data communication with an IC card held by the user. When the user holds the IC card near the communication circuit 60a, the short-range communication circuit 60 reads various data recorded on the IC card.

[0016] The engine control unit 30 controls the scanner unit 31 and the printer unit 32. The operation panel 40 includes a panel display unit 40a such as a touch panel that displays current setting values, selection screens, etc. and accepts inputs from the operator, and an operation panel 40b including a numeric keypad that accepts setting values for conditions related to image formation such as density setting conditions and a start key that accepts a copy start instruction. The controller 20 controls the entire image forming apparatus 10, for example, controls drawing, communication, inputs from the operation panel 40, etc. The scanner unit 31 or the printer unit 32 includes an image processing part such as error diffusion and gamma conversion. In addition, the image forming apparatus 10 can sequentially switch and select a document box function (= scanner function), a copy function, a printer function, and a facsimile function by an application switching key on the operation panel 40. The document box function (scanner function) also has a function of extracting text from an image by OCR and converting it into a format usable in various office applications. The network I / F 50 is an interface for data communication using the communication network N. The short-range communication circuit 60 and the network I / F 50 are electrically connected to the ASIC 26 via the PCI bus 23.

[0017] <Hardware Configuration of the PC> FIG. 3 is a hardware configuration diagram of the PC shown in FIG. 1. Here, the hardware configuration of the PC will be described. As shown in FIG. 3, the PCs 80, 90 are constructed by a computer and include a CPU 201, a ROM 202, a RAM 203, an HD 204, an HDD (Hard Disk Drive) controller 205, a display 206, an external device connection I / F (Interface) 208, a network I / F 209, a data bus 210, a keyboard 211, a pointing device 212, a DVD-RW (Digital Versatile Disk Rewritable) drive 214, and a media I / F 216, as shown in FIG. 3. Among these, the CPU 201 controls the operation of the entire PCs 80 and 90. The ROM 202 stores programs used to drive the CPU 201 such as the IPL. The RAM 203 is used as a work area for the CPU 201. The HD 204 stores various data such as programs. The HDD controller 205 controls the reading or writing of various data to and from the HD 204 according to the control of the CPU 201.

[0018] The display 206 displays various information such as a cursor, menu, window, characters, or images. The external device connection I / F 208 is an interface for connecting various external devices. External devices in this case are, for example, a USB (Universal Serial Bus) memory, printer, etc. The network I / F 209 is an interface for data communication using the communication network N. The bus line 210 is an address bus, data bus, etc. for electrically connecting each component such as the CPU 201 shown in FIG. 3. Also, the keyboard 211 is a type of input means having a plurality of keys for inputting characters, numerical values, various instructions, etc. The pointing device 212 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The DVD-RW drive 214 controls the reading or writing of various data to and from the DVD-RW 213 as an example of a removable recording medium. Note that it is not limited to DVD-RW and may be DVD-R or the like. The media I / F 216 controls the reading or writing (storage) of data to and from the recording medium 215 such as a flash memory.

[0019] <Functional Configuration of Image Processing Apparatus> FIG. 4 is a diagram showing an example of the functional configuration of an image processing apparatus configured by the PC shown in FIG. 3. The image processing apparatus 120 has a function of creating a character code and detecting a font size from the image data of a document, and includes an image acquisition unit 125, an object recognition unit 130, a font size determination unit 140, and an electronic data generation unit 150. For example, a program for executing the processes described below is installed in the HD104 of a personal computer, and it is configured by the CPU executing the program read from the HD104. Note that the image processing apparatus 120 may be configured by installing a program for executing the processes described below in the HD29 of the image forming apparatus 10 shown in FIG. 2, and the CPU 25a executing the program read from the HD29.

[0020] The image acquisition unit 120 acquires the image data of a document from an external terminal via a LAN or from a portable storage medium such as a USB memory. It may be configured to read a document including a document from the scanner unit 31 of an external image processing apparatus 10 via a communication network N or using an external camera to acquire image data.

[0021] The object recognition unit 130 expands the acquired image data of the document in a page memory and recognizes an object. Objects include characters, tables, pictures, graphics, and the like. A known technique is used as means for recognizing such an object. For example, edge enhancement processing is performed on the image data, pixels are scanned in the vertical and / or horizontal directions to detect edges, and a region where the detection frequency is higher than a certain threshold value can be determined as a character object. Also, for the extraction of a table, the region of the table can be extracted as a table object by the method described in Japanese Patent No. 5153857. The other regions can be determined as picture or graphic objects. For the image data of the character object, known character recognition processing is executed to generate text data. For example, for the image data of a character object, create density histograms in the horizontal and vertical directions, cut out the image character by character, extract feature points (such as the number and positions of closed loops and isolated points, etc.) from each cut-out character image, and perform matching with the character patterns stored in a previously prepared table to identify the character code.

[0022] The object recognition unit 130 transmits the character code obtained by performing character recognition processing on the character object, together with its position information, to the electronic data generation unit 150 as character code data. For object of a table, picture, or figure, the image data within its area and the position information (memory address) are transmitted to the electronic data generation unit 150. Also, the character code data, the image data of the character object, and the image data of the table object are transmitted to the font size determination unit 140.

[0023] The font size determination unit 140 determines the font size for each character based on the image data of the character object and the image data of the table object transmitted from the object recognition unit 130. The electronic data generation unit 150 associates the font size and the character code of each character based on their position information so that the characters of the character code created by the object recognition unit 130 can be displayed in the font size determined by the font size determination unit 140, and integrates them with the table, picture, and figure objects of the image data acquired by the image acquisition unit 120 to generate electronic data in a format that can be edited by a predetermined editing application.

[0024] Each function of the embodiment described above can be realized by one or more processing circuits. Here, the "processing circuit" in this specification refers to a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, an ASIC (Application Specific Integrated Circuit) designed to execute each function described above, a DSP (digital signal processor), an FPGA (field programmable gate array), and devices such as conventional circuit modules.

[0025] <Configuration of Font Size Determination Unit> FIG. 5 is a functional block diagram showing the detailed configuration of the font size determination unit 140 shown in FIG. 4. The font size determination unit 140 includes a font size provisional determination unit 141, an in-table character determination unit 142, a cell attribute detection unit 143, a font size group classification unit 144, and a font size final determination unit 145. The font size provisional determination unit 141 provisionally determines the font size for each character based on the image data of the character object received from the object recognition unit 130. For example, the technique described in Patent Document 1 is used as the means for provisionally determining the font size. In Patent Document 1, an enclosing rectangle is obtained for each character, and the font size is detected based on a predetermined standard. Further, by taking into account the information of the characters (adjacent characters, words) around the target character and adjusting the font size, the font size can be unified in units of words or sentences.

[0026] The in-table character determination unit 142 determines whether the character is inside the table based on the position information of the character object and the position information of the table object. The cell attribute detection unit 143 detects the attributes of the character images in each cell of the table object. The detected cell information includes at least the type of character system such as English letters, numbers, Japanese characters (Chinese characters, kana), the type of character alignment such as centered alignment, left alignment, right alignment, and the type of background color. When detecting the type of character system, it can be determined based on the character code data. Also, for the type of character alignment, the cell attribute detection unit 143 can create a density histogram in the cell and determine it from the degree of deviation. For the type of background color, the cell attribute detection unit 143 calculates, for example, the average value of the RGB values of the part excluding the character part in the character image in the cell, and determines the color with the smallest Euclidean distance among a plurality of standard colors (10 colors) defined in Microsoft Office (registered trademark) as the background color.

[0027] The font size group classification unit 144 classifies the font size groups within the table object based on the attributes of the character images in each cell and the temporarily determined font size. The detailed classification method is shown in FIG. 6. The font size final determination unit 145 finally determines the font size based on the font size determined by the font size temporary determination unit, the table character determination result determined by the in-table character determination unit, and the font size group information classified by the font size group classification unit. If it is not an in-table character, the temporarily determined font size is taken as the final determination. If it is determined to be an in-table character, it is determined as the size that is the most frequent among the temporarily determined font sizes of the characters belonging to the same classification from the font size group information. However, if they are the same number, the average value is used.

[0028] <Operation of the Font Size Determination Unit> FIG. 6 is a flowchart showing the operation of determining the font size in the font size determination unit 140 shown in FIG. 5. In step S10, the font size temporary determination unit 141 temporarily determines the font size based on the character object data and the character code data received from the object recognition unit 130. In step S20, the in-table character determination unit 142 determines whether the character object of interest is an in-table character based on the character object data and table object data received from the object recognition unit 130. If the determination result indicates that the character object is an in-table character, the process proceeds to S20; if it is not an in-table character object, the process proceeds to S30.

[0029] In step S30, after all character object determinations are completed in the in-table character determination unit 142, the cell attribute detection unit 143 detects cell attributes based on the in-cell character objects and character code data of the character images within each cell. The cell attribute detection unit 143 detects an attribute representing the characteristics of the character image included in the cell of interest. The cell attribute detection unit 143 uses any one of the type of character system, the type of background color, and the type of character alignment as the attribute of the character image within the cell. Based on the color components of the character image within the cell of interest, the cell attribute detection unit 143 assigns the color with the smallest Euclidean distance among a plurality of standard colors defined in Microsoft Office (registered trademark) as the type of background color.

[0030] In step S40, the font size group classification unit 144 classifies each character object into the font size group it belongs to based on the temporarily determined font size and cell attribute information. The font size group classification unit 144 classifies all the in-cell character images included in the image data based on the attribute of the in-cell character image and the temporarily determined font size for the in-cell character image. When all cell attributes are complete, the cells are adjacent, and the difference between the temporarily determined font sizes of the characters in the adjacent cell and the character of interest is less than or equal to a predetermined value, the font size group classification unit 144 uses the same classification result. When all cell attributes are complete, the cells are adjacent, and the difference between the temporarily determined font sizes of the characters in the adjacent cell and the character of interest is less than or equal to a predetermined value, the font size group classification unit 144 uses the same classification result. For characters not classified in the classification operation, when the type of character system and the type of alignment among the cell attributes are the same, the cells are adjacent, and the difference between the temporarily determined font size and the characters in the adjacent cells and the target character is equal to or less than a predetermined value, the same classification result is used. For characters not classified in the classification operation, the font size determined temporarily is directly determined as the font size of the target character.

[0031] In step S50, in the font size final determination unit 145, the character objects classified into the same font size group are unified to the most frequent font size in that font size group, and the font size is finally determined. The font size final determination unit 145 finally determines the font size for each classification result based on the classification result by the font size group classification unit 144 and the temporarily determined font size. The font size final determination unit 145 finally determines the font size to be the most frequent font size among the font sizes for each classification result by the font size group classification unit 144. When there are the same number of the most frequent font sizes among the font sizes for each classification result by the font size group classification unit 144, the font size final determination unit 145 finally determines the font size to be the average font size. In step S60, in the in-table character determination unit 142, for character objects determined not to be in the table, the temporarily determined font size is directly finally determined.

[0032] <Classification method for each cell> Figs. 7(a) to 7(d) are diagrams showing specific examples of the classification method of the character images in each cell based on the cell attributes. Fig. 7(a) is a diagram showing a document in which image data is printed on a recording medium. In Fig. 7(a), as the names of the respective parts of a general table, on the uppermost row from left to right, the table side head HST, the table head HD1, the table head HD3, and the table head HD3 are arranged. In the table shown in Fig. 7(a), on the uppermost row from left to right, the characters "Food Name", "Protein", "Lipid", and "Carbohydrate" are printed. Also, as the names of the respective parts of a general table, on the leftmost column from top to bottom, the table side head HST, the table side HS1, the table side HS2, and the table side HS3 are arranged. In the table shown in Fig. 7(a), on the leftmost column from top to bottom, the characters "Food Name", "Milk", "Full-fat Milk", and "Low-fat Milk" are printed.

[0033] Fig. 7(b) is a diagram showing the font of the manuscript printed on the recording medium and the font size of that font. Even after generating the electronic data, it is ideal to be in the state shown in Fig. 7(b). Note that in Fig. 7(b), the font size of the font of the manuscript is noted within ().

[0034] Fig. 7(c) is a diagram showing an example after the font size determination unit 141 tentatively determines the font size. Also in Fig. 7(c), the font size is noted within (). Further, the underline " _ " within () shown in Fig. 7(c) is the location where the difference from the font size of the manuscript font appears. In some parts (where there is _), the font size of the manuscript font cannot be accurately reproduced. When the font size determination unit 141 tentatively determines the font size, the font size is determined considering only adjacent characters, so the font size of the character images within the cell can be unified. However, since the font sizes of the character images in other cells are not considered, there are cases where the font sizes vary between cells. For example, as shown in Fig. 7(c), the numbers in the lipid column are only pt20 for "40", which is different from the font sizes of the upper and lower numerical values. In a normal office document, if they are of the same type of character alignment, the same type of character system, and the same type of background color, it is more natural and there is less sense of discomfort when visually viewing if the font sizes are the same.

[0035] <Classification Rules> Next, the font size group classification unit 144 will explain the classification rules for font size groups based on cell attributes and the temporarily determined font size. The classification rules are as follows. The font size group classification unit 144 classifies character objects into font size groups in the following three steps [1] to [3]. [1] Character objects that meet the following conditions are grouped into the same font size group. (a) The type of character system (English, numbers, Japanese characters), the type of background color, and the type of character alignment are the same. (b) They are adjacent. (c) The difference between the temporarily determined font size and the font size of the character object in the adjacent cell is less than or equal to a predetermined value.

[0036] [2] For character images within a cell that were not classified in [1], character objects that meet the following conditions are grouped into the same font size group. (d) The type of character system (English, numbers, Japanese) and the type of character alignment are the same. (e) They are adjacent. (f) The difference between the temporarily determined font size and the font size of the character object in the adjacent cell is less than or equal to a predetermined value.

[0037] [3] Cells that were not classified in [1] and [2] are grouped into other font size groups. According to the above classification rules, the result of classifying Fig. 7(c) is Fig. 7(d). Note that the difference in font size in (c) of [1] and (f) of [2] is set to 2pt here. Groups I to IV are classified at the stage of [1]. At the stage of [2], the V group is classified, and at [3], the VI group is classified. For example, the character object in the III group in [1] has a character system type of numbers, a background color type of light gray, and a character alignment type of right alignment, and thus belongs to the same group. Comparing the III group and the IV group, the character system type and the character alignment type are the same and adjacent, and the difference in font size is also 2 pt or less. However, since the background color types are different, the III group and the IV group are different groups.

[0038] In the font size determination unit 145, the size of the group is unified to the most common size among the font sizes for each group. However, when the number of sizes in the group is the same, the size of the group is unified to the average size. For example, in the case of the I group, since the most common sizes of 20, 16, and 18 pt are the same in number, it is corrected to the average size of 18 pt, and the font size is finally determined.

[0039] <Processing result according to the prior art> FIG. 8 is a diagram showing an example of the processing result by a conventional image processing apparatus in tabular form. In a conventional image processing apparatus, Table J1 is generated as a processing result, and within the frame J3 included in Table J1, four cells arranged horizontally are included. In the four cells within the frame J3, character images with different font sizes are respectively accommodated. That is, according to the conventional processing result, character images with different font sizes are determined in cell units. In a conventional image processing apparatus, "adjacent" characters and words are used as the area for determining the font size from the image data of the manuscript. Also, when the image data includes cells, the font size is determined in cell units.

[0040] <Processing result according to the present invention> FIG. 9 is a diagram showing the recognition result by the image processing apparatus of the present invention in tabular form. In the image processing apparatus of the present invention, Table J5 is generated as a processing result, and within frame J7 included in Table J5, four cells arranged horizontally are included. In the four cells within frame J7, character images with the same font size are respectively accommodated. That is, according to the processing result of the present invention, for the character images within the cells included in the same classification, character images with the same font size are determined.

[0041] In the image processing apparatus of the present invention, the character images within each cell in the same table are classified, and the font size is determined for each classification, so that unintended font size mismatches within the same table can be eliminated. Note that for classification, any one of the type of character system, the type of background color, and the type of character alignment, which are the attributes of the character images within each cell, is used. Here, it is premised that for those character images within the cells whose attributes are more similar, the font sizes are the same. The present invention has the following features when determining the font size of the characters within the table format. Temporarily determine the font size of the character image within a single cell, detect the attributes of the character image within a single cell, classify all cells based on the attributes of the character image within the cell and the temporarily determined font size, and determine the font size for each classification based on the classification and the font size of the single cell. In short, it is characterized in that the font size is determined by the font size of the character image within a single cell and the classification of the cell. According to the present invention, each cell in the same table is classified, and the font size is determined for each classification, so that unintended font size mismatches within the same table can be eliminated. For classification, any one of the type of character system, the type of background color, and the type of character alignment, which are the attributes of each cell, is used.

[0042] <Summary of the Actions and Effects of the Embodiment Example> <First Aspect> The image processing apparatus 120 according to this aspect is an image processing apparatus that identifies a character code corresponding to a character image in each of a plurality of cells from image data related to a document including character images in the plurality of cells, and determines a font size corresponding to the character code. The image processing apparatus includes: a font size provisional determination unit 141 that provisionally determines the font size of a target cell based on character images in cells adjacent to the target cell; a cell attribute detection unit 143 that detects an attribute representing a feature of the character image included in the target cell; a font size group classification unit 144 that classifies character images in all cells included in the image data based on the attribute of the character image in the cell and the font size provisionally determined for the character image in the cell; and a font size final determination unit 145 that determines the font size for each classification result based on the classification result by the font size group classification unit 144 and the provisionally determined font size. According to this aspect, based on the character images in the cells adjacent to the target cell, the font size of the target cell is provisionally determined, the attribute representing the feature of the character image included in the target cell is detected, and based on the attribute of the character image in the cell and the font size provisionally determined for the character image in the cell, all the character images in the cells included in the image data are classified, and based on the classification result and the provisionally determined font size, the font size for each classification result can be determined. Thereby, the font sizes corresponding to the character images in non-adjacent cells can be unified.

[0043] <Second Aspect> The cell attribute detection unit 143 according to this aspect is characterized in that any one of the type of character system, the type of background color, and the type of character alignment is used as the attribute of the character image in the cell. According to this aspect, the cell attribute detection unit 143 can detect the attribute representing the feature of the character image included in the target cell by using any one of the type of character system, the type of background color, and the type of character alignment as the attribute of the character image in the cell.

[0044] <Third Aspect> The cell attribute detection unit 143 of this aspect is characterized in that, as the type of the background color, it assigns the color with the smallest Euclidean distance among a plurality of standard colors defined in Microsoft Office (registered trademark) based on the color components of the character image in the cell of interest. According to this aspect, the cell attribute detection unit 143 can detect an attribute representing the characteristics of the color components of the character image included in the cell of interest by assigning the color with the smallest Euclidean distance among a plurality of standard colors defined in Microsoft Office (registered trademark) as the type of the background color based on the color components of the character image in the cell of interest.

[0045] <Fourth Aspect> The font size group classification unit 144 of this aspect is characterized in that, when all cell attributes are complete, and the cells are adjacent cells, and the difference between the temporarily determined font size of the character in the adjacent cell and the font size of the character of interest is equal to or less than a predetermined value, the same classification result is used. According to this aspect, the font size group classification unit 144 can classify the character image in the cell by using the same classification result when all cell attributes are complete, and the cells are adjacent cells, and the difference between the temporarily determined font size of the character in the adjacent cell and the font size of the character of interest is equal to or less than a predetermined value.

[0046] <Fifth Aspect> The font size group classification unit 144 of this aspect is characterized in that, for the characters not classified in the classification operation, when the type of the character system and the type of alignment among the cell attributes are complete, and the cells are adjacent cells, and the difference between the temporarily determined font size of the character in the adjacent cell and the font size of the character of interest is equal to or less than a predetermined value, the same classification result is used. According to this aspect, the font size group classification unit 144 can classify the character image in the cell by using the same classification result for the characters not classified in the classification operation when the type of the character system and the type of alignment among the cell attributes are complete, and the cells are adjacent cells, and the difference between the temporarily determined font size of the character in the adjacent cell and the font size of the character of interest is equal to or less than a predetermined value.

[0047] <Sixth Aspect> The font size group classification unit 144 in this aspect is characterized in that for characters not classified in the classification operation, the temporarily determined font size is directly determined as the font size of the target character. According to this aspect, the font size group classification unit 144 can determine the font size for characters not classified in the classification operation by directly determining the temporarily determined font size as the font size of the target character.

[0048] <Seventh Aspect> The font size final determination unit 145 in this aspect is characterized in that it determines the most frequent font size among the font sizes for each classification result. According to this aspect, the font size final determination unit 145 can determine the font size by determining the most frequent font size among the font sizes for each classification result.

[0049] <Eighth Aspect> The font size final determination unit 145 in this aspect is characterized in that when the number of the most frequent font sizes among the font sizes for each classification result is the same, it determines the average font size. According to this aspect, the font size final determination unit 145 can determine the font size by determining the average font size when the number of the most frequent font sizes among the font sizes for each classification result is the same.

[0050] <Ninth Aspect> The image processing method of this aspect is an image processing method for specifying character codes corresponding to character images in each of a plurality of cells from image data related to a document containing character images in the plurality of cells, and determining a font size corresponding to the character codes, including: a font size provisional determination step (S10) for provisionally determining the font size of a target cell based on the character images in the cells adjacent to the target cell; a cell attribute detection step (S30) for detecting an attribute representing the characteristics of the character image included in the target cell; a font size group classification step (S40) for classifying the character images in all the cells included in the image data based on the attribute of the character image in the cell and the font size provisionally determined for the character image in the cell; and a font size final determination step (S50) for determining the font size for each classification result based on the classification result by the font size group classification step (S40) and the provisionally determined font size. According to this aspect, based on the character images in the cells adjacent to the target cell, the font size of the target cell is provisionally determined, the attribute representing the characteristics of the character image included in the target cell is detected, the character images in all the cells included in the image data are classified based on the attribute of the character image in the cell and the font size provisionally determined for the character image in the cell, and the font size for each classification result can be determined based on the classification result and the provisionally determined font size. Thereby, the font sizes corresponding to the character images in non-adjacent cells can be unified.

[0051] <Aspect 10> The program of this aspect causes a processor to execute each step in the image processing method described in Aspect 9. According to this aspect, each step can be executed by a processor.

Explanation of Signs

[0052] 1... System, 10... Image processing apparatus, 20... Controller, 22a... ROM, 22b... RAM, 23... PCI bus, 25a... CPU, 26... ASIC, 28... HDD controller, 30... Engine control unit, 31... Scanner unit, 32... Printer unit, 40... Operation panel, 40a... Panel display unit, 40b... Operation panel, 50... Network I / F, 60... Short-range communication circuit, 60... Short-range communication circuit, 60a... Communication circuit, 80... PC, 90... PC, 101... CPU, 102... ROM, 103... RAM, 105... Controller, 105... HDD controller, 106... Display, 108... External device connection I / F, 109... Network I / F, 120... Image processing apparatus, 125... Image acquisition unit, 111... Keyboard, 112... Pointing device, 115... Recording medium, 116... Media I / F, 120... Image acquisition unit, 130... Object recognition unit, 140... Font size determination unit, 141... Temporary font size determination unit, 142... In-table character determination unit, 143... Cell attribute detection unit, 144... Font size group classification unit, 145... Final font size determination unit, 150... Electronic data generation unit, 208... Controller

Prior Art Documents

Patent Documents

[0053]

Patent Document 1

Claims

1. An image processing apparatus that identifies a character code corresponding to a character image in each of a plurality of cells from image data related to a document containing character images in the plurality of cells, and determines a font size corresponding to the character code, the apparatus comprising: a font size provisional determination unit that provisionally determines the font size of the cell of interest based on character images in cells adjacent to the cell of interest; a cell attribute detection unit that detects an attribute representing a feature of the character image included in the cell of interest; a font size group classification unit that classifies character images in all cells included in the image data based on the attribute of the character image in the cell and the font size provisionally determined for the character image in the cell; a font size final determination unit that determines a font size for each classification result based on the classification result by the font size group classification unit and the provisionally determined font size. The image processing apparatus is characterized by comprising the above components.

2. The image processing apparatus according to claim 1, wherein the cell attribute detection unit uses any one of a character system type, a background color type, and a character alignment type as an attribute of the character image in the cell.

3. The image processing apparatus according to claim 2, wherein, as the background color type, the cell attribute detection unit assigns, based on the color components of the character image in the cell of interest, the color having the smallest Euclidean distance among a plurality of standard colors defined in Microsoft Office (registered trademark).

4. The image processing apparatus according to claim 1, wherein the font size group classification unit gives the same classification result when the cell attributes are all the same, the cells are adjacent, and the difference between the provisionally determined font size of the character in the adjacent cell and the font size of the character of interest is equal to or less than a predetermined value.

5. The image processing apparatus according to claim 4, wherein, for a character not classified in the classification operation, the font size group classification unit gives the same classification result when the character system type and the alignment type among the cell attributes are the same, the cells are adjacent, and the difference between the provisionally determined font size of the character in the adjacent cell and the font size of the character of interest is equal to or less than a predetermined value.

6. The font size group classification unit determines, as the font size of the character of interest, the font size tentatively determined as it is for characters not classified in the classification operation, in the image processing apparatus according to claim 4 or 5, which is characterized in that.

7. The font size final determination unit determines the most frequent font size among the font sizes for each classification result, in the image processing apparatus according to claim 1, which is characterized in that.

8. The font size final determination unit determines the average font size when the number of the most frequent font sizes among the font sizes for each classification result is the same, in the image processing apparatus according to claim 1, which is characterized in that.

9. An image processing method for identifying a character code corresponding to a character image in each of a plurality of cells and determining a font size corresponding to the character code from image data related to a document including the character images in the plurality of cells respectively, A font size tentative determination step of tentatively determining the font size of the cell of interest based on the character images in the cells adjacent to the cell of interest, A cell attribute detection step of detecting an attribute representing the characteristics of the character image included in the cell of interest, A font size group classification step of classifying the character images in all the cells included in the image data based on the attribute of the character image in the cell and the font size tentatively determined for the character image in the cell, A font size final determination step of determining the font size for each classification result based on the classification result by the font size group classification step and the font size tentatively determined, characterized by comprising an image processing method.

10. A program characterized by causing a processor to execute each step in the image processing method according to claim 9.

Images