Image processing apparatus, image processing method, and program

Abstract

To improve accuracy in the process of discriminating the upright direction of a character image included in image data.SOLUTION: The present invention comprises: an area division unit 70b for setting upper-limit values to the size in the vertical and horizontal directions for each character image included in image data and dividing into rectangular areas within the upper-limit values; a certainty factor calculation unit 70d for performing character recognition on a character image within the rectangular area along the positive and negative directions of the vertical direction and the positive and negative directions of the horizontal direction, respectively, and calculating certainty factors in four directions; an upright direction determination unit 70e for determining that a direction having a largest certainty factor among the certainty factors in four directions calculated by the certainty factor calculation unit 70d is an upright direction; and a control unit 70. When the upright direction determination unit 70e determines an upright direction, the control unit 70 exercises control so that the upper-limit value of the size set to the area division unit 70b is increased to be greater than when regular character recognition is performed.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, an image processing method, and a program.

Background Art

[0002] Conventionally, in one method for character recognition processing, an image is read using a high-resolution photoelectric conversion means with a large number of pixels and a low-resolution photoelectric conversion means with a small number of pixels. Here, when the object to be read is small characters, the image data read by the high-resolution photoelectric conversion means is used, while when the object to be read is large characters, the image data read by the low-resolution photoelectric conversion means is used for character recognition. Patent Document 1 aims to perform character recognition processing for characters of various character sizes with a simple configuration and high accuracy. It determines whether the size of the character to be subjected to character recognition processing in the image data is larger than a predetermined size. If it is determined to be larger than the predetermined size, at least the region including the above characters in the image data is reduced so that the size of the characters becomes equal to or less than the predetermined size, and character recognition processing of the above characters is performed using the reduced image data. A technique is disclosed. Thus, Patent Document 1 is character recognition processing focusing on character size, and the processing amount has increased due to the accompanying reduction processing.

[0003] By the way, for a character image scanned and read, first, since the upright direction of the character image is unknown, it is necessary to identify the upright direction of the character image, and then perform normal character recognition processing along the identified upright direction. Conventionally, in the upright direction determination processing, for example, there is a method of performing OCR (Optical Character Recognition) processing while rotating (some of) the character images included in the scanned document by 90 degrees each, and setting the direction (the direction in which it can be read) that gives the highest recognition confidence as the upright direction. In the upright direction determination process using such OCR processing, since the processing amount increases when performing OCR processing on the entire character image included in the scanned image, it is common to cut out a part from the entire character image and perform the upright direction determination process on the cut-out partial image. However, in Patent Document 1, it was not a character recognition process premised on identifying the upright direction of characters (the direction in which the characters can be read or the direction in which the subject is upright). That is, in Patent Document 1, since character recognition processing was performed using reduced image data, there was a problem that the accuracy of the upright direction determination process decreased.

Summary of the Invention

Problems to be Solved by the Invention

[0004] One embodiment of the present invention has been made in view of the above, and its object is to improve the accuracy in the process of identifying the upright direction of character images included in image data.

Means for Solving the Problems

[0005] In order to solve the above problems, the invention according to claim 1 is an image processing apparatus that identifies the upright direction of character images included in image data and performs normal character recognition processing along the upright direction, the image processing apparatus including: a region division unit that sets an upper limit value for the sizes in the vertical direction and the horizontal direction for each character image included in the image data and divides the image data into rectangular regions within the upper limit value; a confidence calculation unit that performs character recognition processing along the + direction and the - direction in the vertical direction and the + direction and the - direction in the horizontal direction for the character images within the rectangular regions and calculates the confidence levels in the four directions; an upright direction determination unit that determines, as the upright direction, the direction having the maximum confidence level among the confidence levels in the four directions calculated by the confidence calculation unit; and a control unit, wherein when the upright direction determination unit determines the upright direction, the control unit controls such that the upper limit value of the size set for the region division unit is larger than when performing the normal character recognition processing.

Effects of the Invention

[0006] According to the present invention, it is possible to improve the accuracy in the process of identifying the upright direction of character images included in image data.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Embodiments for Carrying Out the Invention

[0008] Hereinafter, the present invention will be described in detail with reference to the embodiments shown in the drawings. The present invention has the following configuration in order to improve the accuracy in the process of identifying the upright direction of character images included in image data. That is, the image processing apparatus of the present invention is an image processing apparatus that identifies the upright direction of a character image included in image data and performs normal character recognition processing along the upright direction. For each character image included in the image data, an upper limit value is set for the sizes in the vertical direction and the horizontal direction, and the region is divided into rectangular regions within the upper limit value. A region division unit, character recognition processing is performed for the character image within the rectangular region along the + direction and the - direction in the vertical direction, and the + direction and the - direction in the horizontal direction, respectively, and a confidence calculation unit that calculates the confidence levels in four directions, and an upright direction determination unit that determines the direction having the maximum confidence level among the confidence levels in the four directions calculated by the confidence calculation unit as the upright direction, and a control unit. The control unit controls so that the upper limit value of the size set in the region division unit becomes larger when the upright direction determination unit determines the upright direction than when performing normal character recognition processing. By providing the above configuration, it is possible to improve the accuracy in the process of identifying the upright direction of the character image included in the image data. The features of the present invention described above will be described in detail with reference to the following drawings. However, the components, types, combinations, shapes, relative arrangements, etc. described in this embodiment are merely illustrative examples and not intended to limit the scope of the present invention only thereto without specific description. Regarding the features of the present invention described above, it will be described in detail below with reference to the drawings.

[0009] <System Configuration> FIG. 1 is a diagram illustrating the configuration of a system including an image processing apparatus according to an embodiment of the present invention. System 1 includes an image forming apparatus 10 and two personal computers, PC80 and PC90, and these are configured to transmit and receive data to and from each other via a communication network N. The image forming apparatus 10 is an MFP (Multifunction Peripheral) equipped with a scanner unit 31 for reading a document sheet, a printer unit 32 for forming an image on a sheet, and a communication function. When an ADF (Auto Document Feeder) is provided, the scanner unit 31 reads the document sheet set on the ADF. By using the scanner unit 31 and the printer unit 32, the image forming apparatus 10 provides users with functions of copy, printer, scanner, and facsimile. Also, by receiving instructions from a user via the operation panel 40 or receiving instructions from the PCs 80 and 90 via the communication network N, the image forming apparatus 10 executes these functions.

[0010] Further, the image forming apparatus 10 uses optical character recognition (hereinafter, simply referred to as "character recognition" or "OCR processing" as necessary) technology to extract text of electronic data from a scanned image read by the scanner unit 31, and has a function of creating an editable document (e.g., a Microsoft Word document) based on the text and the scanned image. Note that the PCs 80 and 90 are terminals (computers) used by general users who use the image forming apparatus 10. The image processing apparatus according to the present invention is constituted by the image forming apparatus 10 shown in FIG. 1, or the PCs 80 and 90. Note that the PCs 80 and 90 are terminals (computers) used by general users who use the image forming apparatus 10.

[0011] <Hardware Configuration of Image Forming Apparatus> FIG. 2 is a diagram showing an example of the hardware configuration of the image forming apparatus 10 shown in FIG. 1. As shown in FIG. 2, the image forming apparatus 10 includes a controller 20, a short-range communication circuit 60, an engine control unit 30, an operation panel 40, and a network I / F (Interface) 50. The controller 20 includes a CPU 25a which is the main component of a computer, a system memory (MEM-P) 22, a north bridge (NB) 25b, a south bridge (SB) 24, an ASIC (Application Specific Integrated Circuit) 26, a local memory (MEM-C) 27 which is a storage unit, an HDD (Hard Disk Drive) controller 28, and an HD 29 which is a storage unit. Also, the NB 25b and the CPU 25a are integrated by a SoC (System on a Chip) 25. The CPU 25a is a control unit that performs overall control of the image forming apparatus 10. The NB 25b is a bridge for connecting the CPU 25a, the MEM-P 22, and the SB 24, and has a memory controller that controls reading and writing to the MEM-P 22, a PCI (Peripheral Component Interconnect) master, and an AGP target.

[0012] The MEM-P 22 consists of a ROM 22a which is a memory for storing programs and data for realizing each function of the controller 20, and a RAM 22b which is used as a memory for developing programs and data and for drawing during memory printing. Note that the programs stored in the RAM 22b may be provided by recording them in an installable or executable format as files on a computer-readable recording medium such as a CD-ROM, CD-R, or DVD.

[0013] SB24 is a bridge for connecting NB25b with a PCI device and a peripheral device. ASIC26 is an IC (Integrated Circuit) for image processing applications having hardware elements for image processing, and has the role of a bridge for connecting the PCI bus 23, the HDD controller 28, and the MEM-C27 respectively. This ASIC26 consists of a PCI target, an AGP master, an arbiter (ARB) forming the core of the ASIC26, a memory controller for controlling the MEM-C27, a plurality of DMACs (Direct Memory Access Controllers) for performing operations such as rotation of image data by means of hardware logic, etc., and a PCI unit for performing data transfer via the PCI bus 23 between the scanner unit 31 and the printer unit 32. Note that interfaces for USB (Universal Serial Bus) and IEEE1394 (Institute of Electrical and Electronics Engineers 1394) may be connected to the ASIC26.

[0014] MEM-C27 is a local memory used as a copy image buffer and a code buffer. HD29 is a storage that pre-stores programs for executing the operations described later and various control data, and is also used for applications such as accumulating image data and accumulating font data and forms used at the time of printing. HD29 controls the reading or writing of data to HD29 according to the control of the CPU25a. In addition, the short-range communication circuit 60 is provided with a communication circuit 60a such as NFC and Bluetooth (registered trademark), and realizes data communication with an IC card held by the user. When the user holds the IC card near the communication circuit 60a, the short-range communication circuit 60 reads various data recorded on the IC card.

[0015] The engine control unit 30 controls the scanner unit 31 and the printer unit 32. The operation panel 40 includes a panel display unit 40a such as a touch panel that displays current setting values, selection screens, etc. and accepts inputs from the operator, and an operation panel 40b including a numeric keypad that accepts setting values for conditions related to image formation such as density setting conditions, and a start key that accepts a copy start instruction. The controller 20 controls the entire image forming apparatus 10, for example, controls drawing, communication, inputs from the operation panel 40, etc. The scanner unit 31 or the printer unit 32 includes an image processing part such as error diffusion and gamma conversion. Note that the image forming apparatus 10 can sequentially switch and select a document box function (scanner function), a copy function, a printer function, and a facsimile function by using the application switching key on the operation panel 40. The document box function (scanner function) also has a function of extracting text from an image by OCR processing and converting it into a format that can be used in various office applications. Also, the network I / F 50 is an interface for data communication using the communication network N. The short-range communication circuit 60 and the network I / F 50 are electrically connected to the ASIC 26 via the PCI bus 23.

[0016] <Hardware Configuration of Personal Computer> FIG. 3 is a hardware configuration diagram of the personal computer shown in FIG. 1. Here, the hardware configuration of the personal computer will be described. As shown in FIG. 3, the personal computer includes a CPU 101, a ROM 102, a RAM 103, an HD 104, an HDD (Hard Disk Drive) controller 105, a display 106, an external device connection I / F (Interface) 108, a network I / F 109, a data bus 110, a keyboard 111, a pointing device 112, a DVD-RW (Digital Versatile Disk Rewritable) drive 114, and a media I / F 116. Among these, the CPU 101 controls the overall operation of the personal computers 80 and 90. The ROM 102 stores programs used for driving the CPU 101 such as the IPL. The RAM 103 is used as a work area for the CPU 101. The HD 104 stores various data such as programs. The HDD controller 105 controls the reading or writing of various data to and from the HD 104 according to the control of the CPU 101.

[0017] The display 106 displays various information such as a cursor, menu, window, characters, or images. The external device connection I / F 108 is an interface for connecting various external devices. External devices in this case are, for example, a USB (Universal Serial Bus) memory, a printer, and the like. The network I / F 109 is an interface for performing data communication using the communication network N. The bus line 110 is an address bus, a data bus, etc. for electrically connecting each component such as the CPU 101 shown in FIG. 3. Also, the keyboard 111 is a type of input means having a plurality of keys for inputting characters, numerical values, various instructions, and the like. The pointing device 112 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like. The DVD-RW drive 114 controls the reading or writing of various data to and from the DVD-RW 113 as an example of a removable recording medium. Note that it is not limited to DVD-RW, and it may be DVD-R or the like. The media I / F 116 controls the reading or writing (storage) of data to and from the recording medium 115 such as a flash memory.

[0018] <Problems in the determination process of the upright direction> Hereinafter, it will be described with reference to FIG. 6. FIGS. 6(a) to (e) are diagrams showing images to be processed by the image processing apparatus according to each embodiment of the present invention. The image data obtained by scanning the paper surface of the document by the scanner unit 31 is multi-valued image data as shown in FIG. 6(a). When performing OCR processing, it is necessary to binarize the multi-valued image data of the document as shown in Fig. 6(a) using a predetermined threshold value to obtain binary image data as shown in Fig. 6(b). After that, as shown in Fig. 6(c), it is common to perform an orthographic projection or the like in the vertical direction y and the horizontal direction x on the binary image data to extract the rectangular area of the image that is considered to be a character. As shown in Fig. 6(c), if an orthographic projection in the horizontal direction x is performed, the line height Lh can be obtained, so it is possible to cut out in the horizontal direction x. Furthermore, if an orthographic projection in the vertical direction y is performed, the column width can be obtained. Furthermore, as shown in Fig. 6(d), it becomes possible to cut out the rectangular area of the character image, and it becomes possible to separate the character images of "と", "ま", and "と". Then, as shown in Fig. 6(e), it is often assumed that a rectangle with too large a physical length is not a character (ex. a photo image). In this case, an upper limit value is set for the font size included in the document.

[0019] This setting (setting an upper limit value for the font size) is generally correct and has the effect of preventing misrecognition of a photo image as a single character by force. However, in some cases, misrecognition may not be prevented. That is the case when the document contains only unexpectedly large fonts. For example, on slides for presentations, only very large characters may be used sometimes. In such a case, if OCR processing is performed on the premise of a normal font size, an image that is too large for the upper limit value of the rectangle size may be rejected, and OCR processing may not be performed. Certainly, when the document contains only large fonts, since the number of characters contained in the document itself is small, even if OCR processing is not performed, it can be said that the impact itself is small. That is, it is also possible to prioritize the effect of preventing misrecognition of a photo image as a single character by force.

[0020] However, when the OCR process is used not for character recognition itself but for the upright direction determination process, as a result of the above exclusion, the top and bottom margins of the document may not be distinguishable (become indeterminate). When the top and bottom margins of the document cannot be identified, the initial state is assumed to be the upright direction. However, if this assumption is incorrect, a major drawback occurs where the OCR process result for the entire document is incorrect (not comparable to the misrecognition of the photographic image). That is, when performing the OCR process for the upright direction determination process, it is considered better to change the upper limit value of the font size that is the premise. Also, when a high confidence level is obtained by the OCR process depending on the font size changed for the upright direction determination process, it can be understood that the document is likely composed of a large font. Therefore, even when performing the OCR process on the character images of the entire page after the upright direction determination process, it is better to change the above upper limit value.

[0021] That is, the OCR process is performed according to the following procedure. (1) Sample within the page and perform the OCR process, calculate the confidence levels in the +x direction, -x direction, +y direction, and -y direction of the document, and determine that the direction corresponding to the highest confidence level is the upright direction. As a result, the upright direction of the document is obtained. (2) Perform the OCR process along the upright direction for the entire area within the page. In the OCR process, in order to improve the recognition accuracy, an upper limit value is set for the size of the rectangular area described above. That upper limit value needs to be changed for each use case (even in the same OCR process).

[0022] <Functional block diagram> FIG. 4 is a functional block diagram showing the functions of an image processing apparatus according to an embodiment of the present invention. The image processing apparatus is configured, for example, by installing a program for executing the following-described processes in the HD104 of the personal computer shown in FIG. 3 and having the CPU execute the program read from the HD104. Note that the image processing apparatus may be configured by installing a program for executing the processes described below in HD29 of the image forming apparatus 10 shown in FIG. 2 and having the CPU 25a execute the program read from HD29.

[0023] As shown in FIG. 4, the image processing apparatus includes a control unit 70, and the control unit 70 further includes a binary image input unit 70a, a region division unit 70b, a counting unit 70c, a confidence calculation unit 70d, and an upright direction determination unit 70e. The image processing apparatus identifies the upright direction of the character image included in the image data and performs normal character recognition processing along the upright direction. Data obtained by binarizing the multi-valued image data read and acquired by the scanner unit 31 of the document is input to the binary image input unit 70a with a predetermined threshold value. The binary image input unit 70a inputs the binarized data and develops the binary image in the first work area of the RAM 22b.

[0024] The region division unit 70b sets an upper limit value for each character image included in the binary image data in the vertical direction and the horizontal direction, and divides it into rectangular regions within the upper limit value. The counting unit 70c will be described later. The confidence calculation unit 70d performs character recognition processing along the + direction and the - direction in the vertical direction, and the + direction and the - direction in the horizontal direction for the character image within the rectangular region, and calculates the confidence in four directions. The upright direction determination unit 70e determines the direction having the maximum confidence among the confidences in the four directions calculated by the confidence calculation unit 70d as the upright direction. When the upright direction determination unit 70e determines the upright direction, the control unit 70 controls so that the upper limit value of the size set in the region division unit 70b becomes larger than when performing normal character recognition processing. When the actual value of the size does not exceed the upper limit value of the size, when the determination result by the upright direction determination unit 70e becomes indeterminate, the control unit 70 controls so that the upper limit value of the size becomes larger.

[0025] The counting unit 70c counts the number of character images whose actual size exceeds the upper limit value among the rectangular regions divided by the region dividing unit 70b in the rectangular regions within a row or a column. When the number of character images counted by the counting unit 70c is less than or equal to a reference value within a row or a column, the control unit 70 controls to increase the upper limit value of the size.

[0026] The counting unit 70c counts the number of adjacent character images among the character images whose actual size exceeds the upper limit value in the rectangular regions within a row or a column among the rectangular regions divided by the region dividing unit 70b, where the character images are adjacent to each other. The control unit 70 is characterized in that when the number of adjacent character images counted by the counting unit 70c becomes two or more for the first time within a row or a column, it controls to increase the upper limit value of the size.

[0027] When the number of character images counted by the counting unit 70c is more than half within the same row or the same column, the control unit 70 controls to increase the upper limit value of the size.

[0028] The control unit 70 extracts a character image whose actual size exceeds the upper limit value of the size, and when the extracted character images straddle adjacent rows or columns, it controls to inherit the upper limit value of the size.

[0029] The confidence calculation unit 70d performs character recognition processing along at least two directions among the + direction and - direction in the vertical direction and the + direction and - direction in the horizontal direction for the character images within the rectangular region, and calculates the confidence levels in at least two directions. The upright direction determination unit 70e determines, as the upright direction, the direction having a confidence level in which each of the at least two directions calculated by the confidence calculation unit 70d is greater than a predetermined threshold value. When the control unit 70 controls to increase the upper limit value of the size, it controls to increase the threshold value of the confidence level.

[0030] When at least one of the confidence levels in the four directions calculated by the confidence level calculation unit 70d exceeds the first threshold, the upright direction determination unit 70e integrates the confidence level only for the direction with the maximum confidence level, and when the value obtained by dividing the integrated confidence level for each direction by the number of integration times exceeds a second threshold different from the first threshold, the upright direction determination unit 70e sets the direction as the upright direction.

[0031] When the upright direction determination unit 70e determines the upright direction, the control unit 70 controls so that the upper limit value of the size set in the region division unit 70b is larger than when performing normal character recognition processing, and based on the increased upper limit value, controls to determine the upper limit value of the character size in normal character recognition processing.

[0032] Each function of the embodiment described above can be realized by one or more processing circuits. Here, the "processing circuit" in this specification includes a processor programmed to execute each function by software like a processor implemented by an electronic circuit, an ASIC (Application Specific Integrated Circuit) designed to execute each function described above, a DSP (digital signal processor), an FPGA (field programmable gate array), and devices such as conventional circuit modules.

[0033] <First Embodiment> Next, FIG. 5 is a flowchart showing the processing of the image processing apparatus (first aspect, second aspect, seventh aspect, eighth aspect) according to the first embodiment of the present invention. Note that steps S306 and S307 shown in FIG. 5 are the same as the technical matters described in Japanese Patent Application Laid-Open No. 2000-113103. In FIG. 5, the processing of rechecking when it is unidentifiable (steps S307 to S311) is characteristic. In FIG. 5, in step S301, the binary image input unit 70a inputs a binary image. Note that the multi-valued image data obtained by reading and acquiring a document by the scanner unit 31 is multi-valued image data as shown in FIG. 6(a). Data obtained by binarizing this multi-valued image data with a predetermined threshold value is input to the binary image input unit 70a, and the binary image is developed in the first work area of the RAM 22b. In step S302, the region division unit 70b performs a full-line cut-out process on the image developed in the first work area of the RAM 22b, and stores the cut-out image in the second work area of the RAM 22b.

[0034] In step S303, the region division unit 70b excludes rows in the cut-out image in the second work area of the RAM 22b whose height exceeds Lh (upper limit value). In step S304, the region division unit 70b cuts out the rectangular area of characters for all rows after exclusion in the second work area of the RAM 22b, and stores the cut-out image in the third work area of the RAM 22b. In step S305, the region division unit 70b excludes rectangular areas in the third work area of the RAM 22b whose width exceeds Cw (upper limit value). By going through step S305, the rectangular areas of all characters to be subjected to OCR processing included in the input image can be obtained and stored in the third work area of the RAM 22b. In steps S303 to S305, the region division unit 70b can set upper limit values for the vertical and horizontal sizes for each character image included in the image data, and divide them into rectangular areas within the upper limit values.

[0035] <Regarding the Region Division of Binary Images> Here, the region division of the binary image in steps S301 to S302 will be described. FIG. 6(a) is an example of multi-valued image data G1. FIG. 6(b) is an example of a binarized image G2. The multi-valued image data is compared with a threshold value to generate a binarized image. In FIG. 6(c), if an orthographic projection (line segment α1) is taken in the horizontal direction x, the rows can be cut out. In the example shown in FIG. 6(c), the length of the line segment α1 is the height of "ま", which is the row height Lh in this embodiment. Furthermore, in FIG. 6(c), by taking projections in the vertical direction (line segments β1, β2, β3), as shown in FIG. 6(d), the rectangular regions R1, R2, and R3 of the characters can be cut out. In this case, the height of each character is obtained by taking the orthographic projection in the horizontal direction x again.

[0036] The confidence calculation unit 70d performs character recognition processing along the + direction and - direction in the vertical direction, and the + direction and - direction in the horizontal direction for the character images within the rectangular region, and calculates the confidence levels in four directions. After that, in step S306, the upright direction determination unit 70e performs OCR processing in four directions (±x direction, ±y direction) and compares the confidence levels for each character image. When the maximum value of the confidence level (Confidence) is equal to or greater than a predetermined value Cth = 0.8, one count is added in the direction with the highest confidence level (performing the "voting" described in Japanese Patent Application Laid-Open No. 2000-113103), and the count number in each of the four directions is divided by the cumulative count number to normalize the sum of the count numbers to 1.

[0037]

Table 1

[0038] The confidence level referred to here is, similar to the technical matters described in Japanese Patent Application Laid-Open No. 2000-113103, a quantity representing how reliable the result of the OCR processing is, and the confidence level is represented by a numerical value from 0 to 1. Typically, after normalizing the number of pixels of each of the cut-out character images, feature amounts are extracted, and by performing matching between the feature amounts and a pattern dictionary, the similarity (Euclidean distance or Mahalanobis distance) is converted to a value from 0 to 1.

[0039] Then, in step S307, when there are at least three or more count numbers, the upright direction determination unit 70e compares the maximum value obtained by normalizing the cumulative count result with the vertical and horizontal discrimination threshold Dth = 0.85. When the maximum value obtained by normalizing the cumulative count result is greater than or equal to the vertical and horizontal discrimination threshold Dth, the direction of the maximum value is determined as the upright direction, and the determination process of the upright direction is terminated. Here, at least as an initial value, the confidence threshold Cth < the determination threshold Dth of the upright direction determination process (eighth aspect). The confidence regarding a single character is insufficient. On the other hand, since an extremely high confidence is reliable, by setting a threshold (vertical and horizontal discrimination threshold) for the value obtained by normalizing the confidence for a plurality of characters to be larger than the confidence threshold for a single character, the accuracy of the upright direction determination process can be ensured (effect of the eighth aspect).

[0040] When the maximum value obtained by normalizing the cumulative count result is less than the vertical and horizontal discrimination threshold Dth, the process proceeds to step S308, and the upright direction determination unit 70e determines whether there is an unprocessed rectangle remaining. When the normalized maximum value is less than the vertical and horizontal discrimination threshold Dth and there is an unprocessed rectangle remaining (Yes in step S308), the process proceeds to step S309, the rectangular area of the next character is selected, and the process returns to step S306 to repeat the counting. The rectangle selection is performed in raster order. Even if the rectangular areas of the characters in all rows with a height Lh or more are processed, when the number of counts is less than 3 or when the normalized maximum value is less than a predetermined value, the upright direction is indeterminate, and the process proceeds to step S310 (claim 2).

[0041] In step S310, the upright direction determination unit 70e determines whether Lh is less than the image height and Cw (upper limit value) is less than the image width, which is the central part of the present embodiment. If Lh is less than the image height and Cw is less than the image width (Yes in step S310), the process proceeds to step S311, where Lh = Lh * 1.5 and Cw = Cw * 1.5 to increase the upper limit values of the line height Lh and the character width Cw (first aspect), and further Cth = Cth * 1.1 (where the upper limit value of Cth is set to 1) to increase the threshold of the confidence level sufficient for voting (seventh aspect), then returns to step S303, and the processes after step S303 are executed again.

[0042] As a result, a rectangular area of a character larger than the originally assumed size can be made the target of voting, so the cases of being upright and indeterminate can be reduced, and as a result, the accuracy of the upright direction determination process can be improved (effect of the first aspect). Also, since the rectangular size is only increased when the upright direction determination process becomes indeterminate, it has no impact when it is within the assumption, and adaptive processing can be performed only when it is outside the assumption (effect of the second aspect). Note that simultaneously with Cth = Cth * 1.1, Dth may be updated such as Dth = Dth * 1.05 to maintain Cth < Dth as much as possible. On the other hand, if Lh is less than the image height and Cw is less than the image width (No in step S310), the process proceeds to step S312 and finally determines it as unidentifiable.

[0043] Also, in this embodiment, by increasing the threshold of the confidence level because a rectangular area of a character larger than the originally assumed size is made the target of voting, it is possible to avoid the drawback that a rectangle that is actually a photo (a rectangle that does not look like a character = a rectangle with a low confidence level) becomes the target of voting. As a result, in a state where the above drawback is reduced, the upper limit value of the rectangular area of the character can be set large, and as a result, the accuracy of the upright direction determination process can be improved (effect of the seventh aspect).

[0044] <Second Embodiment> Next, FIG. 7 is a flowchart showing the processing of the image processing apparatus (first aspect, third aspect, sixth aspect) according to the second embodiment of the present invention. In FIG. 7, the processing of reviewing when the number of rectangular regions in a line is insufficient (steps S405 to S412) is characteristic. In FIG. 7, in step S401, the binary image input unit 70a inputs a binary image. Note that the multi-valued image data obtained by reading a document with the scanner unit 31 is multi-valued image data as shown in FIG. 6(a), and data obtained by binarizing this multi-valued image data with a predetermined threshold value is input to the binary image input unit 70a, and the binary image is developed in the first work area of the RAM 22b. In step S402, the region division unit 70b performs a full-line cut-out process on the image developed in the first work area of the RAM 22b, and stores the cut-out image in the second work area of the RAM 22b. In step S403, the region division unit 70b initializes the upper limit value Cw of the character width, the upper limit value Ch of the character height, and the threshold value Cth of the confidence level.

[0045] In step S404, the region division unit 70b determines, for each line in the second work area of the RAM 22b, the rectangular regions for all characters to be OCR-processed within one line by excluding rectangular regions where the character width > Cw and the character height > Ch, and stores the image in the third work area of the RAM 22b. In step S405, the region division unit 70b determines whether the number of rectangular regions of characters to be OCR-processed within the line is less than a predetermined value (third aspect). Here, if the number of rectangular regions of characters is not less than the predetermined value, the process proceeds to step S408. On the other hand, if the number of rectangular regions of characters is less than the predetermined value, the process proceeds to step S406. In step S406, the region division unit 70b compares whether Cw is less than 1 / 2 of the image width and whether Ch is less than 1 / 2 of the image width. Here, if Cw is not less than 1 / 2 of the image width or Ch is not less than 1 / 2 of the image width, the process proceeds to step S408. On the other hand, if Cw is less than 1 / 2 of the image width and Ch is less than 1 / 2 of the image width, the process proceeds to step S407.

[0046] In step S407, the region division unit 70b sets Cw = Cw * 1.5, Ch = Ch * 1.5, and Cth = Cth * 1.1 (first aspect). In this way, by previously counting the number of rectangular regions of the characters to be processed within a line and setting a larger upper limit value for the rectangular size when the number of rectangles is small, the number of rectangles sufficient for the upright direction determination process can be ensured in more cases, and the cases of being upright indeterminate can be reduced. As a result, the accuracy of the upright direction determination process can be improved (effect of the third aspect).

[0047] The confidence calculation unit 70d performs character recognition processing along the + direction and - direction in the vertical direction, and the + direction and - direction in the horizontal direction for the character images within the rectangular regions, respectively, and calculates the confidence levels in four directions. After that, in step S408, the upright direction determination unit 70e performs OCR processing in four directions (±x direction, ±y direction) and comparison of confidence levels for each character image unit. When the maximum value of the confidence level (Confidence) is equal to or greater than a predetermined value Cth = 0.8, one count is added to the direction with the maximum confidence level (performing the "voting" described in Japanese Patent Application Laid-Open No. 2000-113103), and the count number in each of the four directions is divided by the cumulative count number to normalize the sum of the count numbers to 1.

[0048] Then, in step S409, the upright direction determination unit 70e compares the maximum value obtained by normalizing the cumulative count result with the top-bottom discrimination threshold Dth = 0.85 when there are at least three or more count numbers. When the maximum value obtained by normalizing the cumulative count result is equal to or greater than the top-bottom discrimination threshold Dth, the direction of the maximum value is determined as the upright direction, and the upright direction determination process is terminated. On the other hand, when the maximum value obtained by normalizing the cumulative count result is less than the top-bottom discrimination threshold Dth, the process proceeds to step S410, and the upright direction determination unit 70e determines whether there are any remaining unprocessed rectangles. When the normalized maximum value is less than the top-bottom discrimination threshold Dth and there are remaining unprocessed rectangles (Yes in step S410), the process proceeds to step S411, the rectangular region of the next character is selected, and the process returns to step S408 to repeat the above counting. The rectangle selection is performed in raster order.

[0049] Also, when the identification is not completed (S409, No), the process proceeds to step S410 to determine whether there is a rectangular area of characters remaining in the line. Here, when there is no remaining rectangular area of characters in the line (S410, No), and when there are still lines remaining in the image (S412, Yes), the process proceeds to step S404. As a result, the updated upper limit values Cw and Ch, which have been updated to larger values, are carried over to the next line (Sixth aspect). By performing carry-over instead of initialization between lines, the upper limit values can be updated efficiently (Effect of the sixth aspect). On the other hand, in step S412, when there are no lines remaining in the image (S412, No), the process proceeds to step S413 and finally determines that it is unidentifiable.

[0050] <Third Embodiment> Next, FIG. 8 is a flowchart showing the processing of the image processing apparatus (First aspect, Fourth aspect) according to the third embodiment of the present invention. In FIG. 8, the processing of rechecking when a continuous rectangular area within a line is excluded (steps S505 to S512) is characteristic. In FIG. 8, in step S501, the binary image input unit 70a inputs a binary image. Note that the multi-valued image data obtained by reading a document with the scanner unit 31 is multi-valued image data as shown in FIG. 6(a), and the data obtained by binarizing this multi-valued image data with a predetermined threshold value is input to the binary image input unit 70a, and the binary image is developed in the first work area of the RAM 22b. In step S502, the region division unit 70b performs a process of cutting out the entire line for the image developed in the first work area of the RAM 22b, and stores the cut-out image in the second work area of the RAM 22b. In step S503, the region division unit 70b initializes the upper limit value Cw of the character width and the upper limit value Ch of the character height.

[0051] In step S504, the region division unit 70b determines, for each row in the second work area of the RAM 22b, the rectangular regions for all the characters to be subjected to OCR processing within one row by excluding rectangular regions where the character width > Cw and the character height > Ch, and stores the images thereof in the third work area of the RAM 22b. In step S505, the region division unit 70b determines whether or not the rectangular regions of adjacent characters within the row have both been excluded (fourth aspect). Here, if the rectangular regions of adjacent characters within the row have not both been excluded, the process proceeds to step S508. On the other hand, if the rectangular regions of adjacent characters within the row have both been excluded, the process proceeds to step S506. In step S506, the region division unit 70b compares whether Cw is less than 1 / 2 of the image width and whether Ch is less than 1 / 2 of the image width. Here, if Cw is not less than 1 / 2 of the image width or Ch is not less than 1 / 2 of the image width, the process proceeds to step S508. On the other hand, if Cw is less than 1 / 2 of the image width and Ch is less than 1 / 2 of the image width, the process proceeds to step S507.

[0052] In step S507, the region division unit 70b sets Cw = Cw * 1.5, Ch = Ch * 1.5, and Cth = Cth * 1.1. In this way, by previously counting the number of rectangular regions of characters to be processed within a row and setting a larger upper limit value for the rectangular size when the number of rectangles is small, the number of cases where a sufficient number of rectangles can be ensured for the upright direction determination process increases, and the number of cases of upright indeterminacy can be reduced. As a result, the accuracy of the upright direction determination process can be improved.

[0053] The confidence calculation unit 70d performs character recognition processing on the character images within the rectangular regions along the + direction and - direction in the vertical direction, and the + direction and - direction in the horizontal direction, respectively, and calculates the confidence levels in the four directions. Thereafter, in step S508, the upright direction determination unit 70e performs OCR processing in four directions (±x direction, ±y direction) for each character image and compares the confidence levels. When the maximum value of the confidence level is equal to or greater than a predetermined value Cth = 0.8, one count is added in the direction with the highest confidence level (performing "voting" described in Japanese Patent Laid-Open No. 2000-113103), and the count number in each of the four directions is divided by the cumulative count number to normalize the sum of the count numbers to 1.

[0054] Then, in step S509, when there are at least three or more count numbers, the upright direction determination unit 70e compares the maximum value obtained by normalizing the cumulative count result with the top-bottom discrimination threshold Dth = 0.85. When the maximum value obtained by normalizing the cumulative count result is equal to or greater than the top-bottom discrimination threshold Dth, the direction of the maximum value is determined as the upright direction, and the determination process for the upright direction is terminated. On the other hand, when the maximum value obtained by normalizing the cumulative count result is less than the top-bottom discrimination threshold Dth, the process proceeds to step S510, and the upright direction determination unit 70e determines whether there are any remaining unprocessed rectangles. When the normalized maximum value is less than the top-bottom discrimination threshold Dth and there are remaining unprocessed rectangles (Yes in step S510), the process proceeds to step S511, the rectangular area of the next character is selected, and the process returns to step S508 to repeat the above counting. The rectangle selection is performed in raster order.

[0055] Also, when the identification is not completed (No in S509), the process proceeds to step S510, and it is determined whether there are any remaining rectangular areas of characters within the line. Here, when there are no remaining rectangular areas of characters within the line (No in S510) and there are still remaining lines in the image (Yes in S512), the process proceeds to step S504. As a result, the updated upper limit values Cw and Ch with larger values are carried over to the next line. By performing carry-over instead of initialization between lines, the upper limit values can be updated efficiently. On the other hand, in step S512, when there are no remaining lines in the image (No in S512), the process proceeds to step S513, and finally, it is determined as unidentifiable.

[0056] In step S505, when two adjacent rectangular regions are excluded, since the actual character size in the image may be larger than expected, by setting the upper limit value of the size of the rectangular region to be larger, the number of rectangles sufficient for the upright direction determination process can be ensured in more cases, and the cases where the upright direction is indeterminate can be reduced. As a result, the accuracy of the upright direction determination process can be improved (the effect of the fourth aspect).

[0057] <Fourth Embodiment> Next, FIG. 9 is a flowchart showing the processing of the image processing apparatus (first aspect, fifth aspect) according to the fourth embodiment of the present invention. In FIG. 9, the processing of rechecking when the majority of rectangular regions in a row are excluded (steps S605 to S612) is characteristic. In FIG. 9, in step S601, the binary image input unit 70a inputs a binary image. Note that the multi-value image data obtained by reading the document by the scanner unit 31 is multi-value image data as shown in FIG. 6(a), and the data obtained by binarizing this multi-value image data with a predetermined threshold value is input to the binary image input unit 70a, and the binary image is developed in the first work area of the RAM 22b. In step S602, the region division unit 70b performs a process of cutting out the entire row for the image developed in the first work area of the RAM 22b, and stores the cut-out image in the second work area of the RAM 22b. In step S603, the region division unit 70b initializes the upper limit value Cw of the character width and the upper limit value Ch of the character height.

[0058] In step S604, the region division unit 70b determines the rectangular regions for all the characters to be OCR-processed in one row by excluding the rectangular regions where the character width > Cw and the character height > Ch in the second work area of the RAM 22b, and stores the image in the third work area of the RAM 22b. In step S605, the region division unit 70b determines whether the rectangular regions of the majority of characters in the line have been excluded (fourth aspect). Here, if the rectangular regions of the majority of characters in the line have not been excluded, the process proceeds to step S608. On the other hand, if the rectangular regions of the majority of characters in the line have been excluded, the process proceeds to step S606. In step S606, the region division unit 70b compares whether Cw is less than 1 / 2 of the image width and whether Ch is less than 1 / 2 of the image width. Here, if Cw is not less than 1 / 2 of the image width or Ch is not less than 1 / 2 of the image width, the process proceeds to step S608. On the other hand, if Cw is less than 1 / 2 of the image width and Ch is less than 1 / 2 of the image width, the process proceeds to step S607.

[0059] In step S607, the region division unit 70b sets Cw = Cw * 1.5, Ch = Ch * 1.5, and Cth = Cth * 1.1. In this way, by counting in advance the number of rectangular regions of characters to be processed in a line and setting a larger upper limit value of the rectangular size when the number of rectangles is small, the number of rectangles sufficient for the upright direction determination process can be ensured in more cases, and the cases of being upright indeterminate can be reduced. As a result, the accuracy of the upright direction determination process can be improved.

[0060] The confidence calculation unit 70d performs character recognition processing along the + direction and - direction in the vertical direction, and the + direction and - direction in the horizontal direction for the character images within the rectangular regions, and calculates the confidence levels in the four directions. Thereafter, in step S608, the upright direction determination unit 70e performs OCR processing in four directions (±x direction, ±y direction) and comparison of confidence levels in units of character images. When the maximum value of the confidence level (Confidence) is equal to or greater than a predetermined value Cth = 0.8, one count is added in the direction with the highest confidence level (performing the "voting" described in Japanese Patent Application Laid-Open No. 2000-113103), and the count number in each of the four directions is divided by the cumulative count number to normalize the sum of the count numbers to 1.

[0061] Then, in step S609, when there are at least three or more count numbers, the upright direction determination unit 70e compares the maximum value obtained by normalizing the cumulative count result with the vertical and horizontal discrimination threshold Dth = 0.85. When the maximum value obtained by normalizing the cumulative count result is greater than or equal to the vertical and horizontal discrimination threshold Dth, the direction of the maximum value is determined as the upright direction, and the determination process of the upright direction is terminated. On the other hand, when the maximum value obtained by normalizing the cumulative count result is less than the vertical and horizontal discrimination threshold Dth, the process proceeds to step S610, and the upright direction determination unit 70e determines whether there is an unprocessed rectangle remaining. When the normalized maximum value is less than the vertical and horizontal discrimination threshold Dth and there is an unprocessed rectangle remaining (Yes in step S610), the process proceeds to step S611, the rectangular area of the next character is selected, and the process returns to step S608 to repeat the counting. The rectangle selection is performed in raster order.

[0062] Also, when the identification is not completed (No in S609), the process proceeds to step S610, and it is determined whether there is a rectangular area of a character remaining in the line. Here, when there is no rectangular area of a character remaining in the line (No in S610) and there is a line remaining in the image (Yes in S612), the process proceeds to step S604. As a result, the updated upper limit values Cw and Ch, which are updated to larger values, are carried over to the next line. By performing carry-over instead of initialization between lines, the upper limit values can be updated efficiently. On the other hand, in step S612, when there is no line remaining in the image (No in S612), the process proceeds to step S613, and finally, it is determined that it is impossible to identify.

[0063] At this time, it is verified whether more than half of the rectangles in the line have been excluded (S605) (the fifth aspect). When more than half of the rectangles in the line have been excluded, it is compared whether Cw is less than 1 / 2 of the image width and Ch is less than 1 / 2 of the image width (S606). When Cw is less than 1 / 2 of the image width and Ch is less than 1 / 2 of the image width, in step S607, Cw = Cw * 1.5, Ch = Ch * 1.5, and Cth = Cth * 1.1 (the first aspect). When more than half of the rectangles are excluded, it is likely that the actual character size in the image is larger than expected. Therefore, by setting a larger upper limit value for the rectangle size, it becomes more likely to ensure a sufficient number of rectangles for the upright direction determination process, and the cases where the upright direction is indeterminate can be reduced. As a result, the accuracy of the upright direction determination process can be improved (the effect of the fifth aspect).

[0064] <Fifth Embodiment> Next, FIGS. 10(a) to 10(d) are flowcharts showing the processing of an image processing apparatus (ninth aspect) according to the fifth embodiment of the present invention. In FIG. 10, the OCR processing for the entire image is executed using the size of the rectangular area during the upright direction determination process. <Basic Example> FIG. 10(a) is a basic example of a flowchart showing the processing of an image processing apparatus (ninth aspect) according to the fifth embodiment of the present invention. In FIG. 10(a), in step S701A, a binary image is input. Next, in step S702A, a full-line extraction process is performed. In step S703A, the control unit 70 performs the upright direction determination process in step S300 described above.

[0065] In step S704A, the values of Lh and Cw after the process in step S300 described above are held. In step S705A, it is determined whether the top and bottom can be identified. Here, if the top and bottom cannot be identified (No), it is assumed that the upright direction is indeterminate, and the process proceeds to step S707A. On the other hand, if the top and bottom can be identified (Yes), it is assumed that the upright direction can be specified, and the process proceeds to step S706A. In step S706A, the entire image is rotated in the identified upright direction, and a full-line extraction process is performed. In step S707A, using the values of Lh and Cw after the process in step S300 described above, the rectangular areas of characters that are not targets of the OCR process are excluded for the entire line. In step S708A, the OCR process is executed for all the remaining rectangular areas of characters after the exclusion.

[0066] In FIG. 10(a), in this embodiment, after executing step S300 in step S703A, the values of Lh and Cw are held (step S704A). When the determination process of the upright direction can be performed (step S705A, Yes), the entire image is rotated in the identified upright direction, and the full-line cutting process is performed (step S706A). Thereafter, using the values of Lh and Cw after the process ends, the rectangular regions of characters outside the OCR process target are excluded for the entire line (step S707A), and the OCR process is executed for all of the rectangular regions of the characters after the exclusion (step S708A) (ninth aspect).

[0067] <Modification Example 1> FIG. 10(b) is a modification example 1 of the flowchart showing the processing of the image processing apparatus (ninth aspect) according to the fifth embodiment of the present invention. Steps S701B to S708B shown in FIG. 10(b) are modifications of steps S701A to S708A shown in FIG. 10(a), respectively. In FIG. 10(b), in this embodiment, after executing step S400 in step S703B, the values of Cw and Ch are held (step S704B). When the determination process of the upright direction can be performed (step S705B, Yes), the entire image is rotated in the identified upright direction, and the full-line cutting process is performed (step S706B). Thereafter, using the values of Cw and Ch after the process ends, the rectangular regions of characters outside the OCR process target are excluded for the entire line (step S707B), and the OCR process is executed for all of the rectangular regions of the characters after the exclusion (step S708B) (ninth aspect).

[0068] <Modification Example 2> FIG. 10(c) is a modification example 2 of the flowchart showing the processing of the image processing apparatus (ninth aspect) according to the fifth embodiment of the present invention. Steps S701C to S708C shown in FIG. 10(c) are modifications of steps S701A to S708A shown in FIG. 10(a), respectively. In FIG. 10(c), in this embodiment, after executing step S500 in step S703C, the values of Cw and Ch are retained (step S704C). If the upright direction determination process is successful (step S705C, Yes), the entire image is rotated in the identified upright direction, and the full-line extraction process is performed (step S706C). Thereafter, using the values of Cw and Ch after the process ends, rectangular regions of characters that are not subject to OCR processing are excluded for the entire line (step S707C), and OCR processing is executed for all of the rectangular regions of the characters after exclusion (step S708C) (the ninth aspect).

[0069] <Modification Example 3> FIG. 10(d) is a modification example 3 of a flowchart showing the processing of an image processing apparatus (the ninth aspect) according to the fifth embodiment of the present invention. Steps S701D to S708D shown in FIG. 10(d) are modifications of steps S701A to S708A shown in FIG. 10(a), respectively. In FIG. 10(d), in this embodiment, after executing step S600 in step S703D, the values of Cw and Ch are retained (step S704D). If the upright direction determination process is successful (step S705D, Yes), the entire image is rotated in the identified upright direction, and the full-line extraction process is performed (step S706D). Thereafter, using the values of Cw and Ch after the process ends, rectangular regions of characters that are not subject to OCR processing are excluded for the entire line (step S707D), and OCR processing is executed for all of the rectangular regions of the characters after exclusion (step S708D) (the ninth aspect).

[0070] In this way, since the upper limit value of the size of the rectangular region of the characters rechecked during the upright direction determination process is also used during the subsequent OCR processing, if the upright direction determination process does not become indeterminate, the rechecked rectangular size is likely to be appropriate, and characters (erroneously) excluded during the subsequent OCR processing can be reduced. As a result, the recognition rate of the OCR processing can be improved (the effect of the ninth aspect). Note that the method of setting an upper limit value for the size of the rectangular area of the characters to be subjected to OCR processing is not limited to the method performed before the OCR processing as in this embodiment, but can also be performed during the OCR processing. That is, rectangles of all sizes are input to the OCR processing main body (e.g., OCR processing library), but the processing of rectangular areas with a predetermined size or more is not performed inside the main body. In such a case, it is common for the upper limit value of the rectangular area to be processed to be passed as a parameter to the OCR processing main body, and it may be configured such that the upper limit value after review is passed as a parameter. Note that FIGS. 5 to 10 are embodiments of the tenth aspect (the first aspect to the ninth aspect), and a system that can achieve effects corresponding to the first aspect to the ninth aspect can be realized (effects of the tenth aspect). Further, from the above description, it is clear that the eleventh aspect can be easily implemented and its effects are also clear.

[0071] <Summary of the operations and effects of the example aspects of this embodiment> <First aspect> The image processing apparatus of this aspect is an image processing apparatus that identifies the upright direction of the character images included in the image data and performs normal character recognition processing along the upright direction. For each character image included in the image data, an upper limit value is set for the size in the vertical direction and the horizontal direction, and a region dividing unit 70b that divides the image into rectangular regions within the upper limit value; character recognition processing is performed for the character images within the rectangular regions along the + direction and the - direction in the vertical direction, and the + direction and the - direction in the horizontal direction, respectively, and a confidence calculation unit 70d that calculates the confidence levels in four directions; an upright direction determination unit 70e that determines the direction having the maximum confidence level among the confidence levels in the four directions calculated by the confidence calculation unit 70d as the upright direction; and a control unit 70. The control unit 70 is characterized in that when the upright direction determination unit 70e determines the upright direction, it controls so that the upper limit value of the size set for the region dividing unit 70b is larger than when performing normal character recognition processing. According to this aspect, since it is possible to control so that the upper limit value of the size set in the region division unit 70b becomes larger, a rectangular region of characters larger than the initially assumed size can be made the target of voting. Therefore, the number of cases where the determination of the upright direction becomes indeterminate can be reduced, and as a result, the accuracy of the process of identifying the upright direction can be improved.

[0072] <Second Aspect> The control unit 70 of this aspect is characterized in that when the actual value of the size does not exceed the upper limit value of the size, and when the determination result by the upright direction determination unit 70e becomes indeterminate, it controls so that the upper limit value of the size becomes larger. According to this aspect, when the determination result by the upright direction determination unit 70e becomes indeterminate, the size of the rectangular region is set large for the first time. Therefore, it has no influence when it is within the initially assumed range, while on the other hand, an adaptive process of controlling so that the upper limit value of the size becomes larger is possible only when it is outside the initially assumed range.

[0073] <Third Aspect> The image processing apparatus of this aspect includes a counting unit 70c that counts the number of character images whose actual size exceeds the upper limit value in the rectangular regions within a row or a column among the rectangular regions divided by the region division unit 70b. The control unit 70 is characterized in that when the number of character images counted by the counting unit 70c is less than or equal to a reference value within a row or a column, it controls so that the upper limit value of the size becomes larger. According to this aspect, when the number of character images counted by the counting unit 70c is less than or equal to a reference value within a row or a column, by controlling so that the upper limit value of the size becomes larger, the number of cases where a sufficient number of rectangular regions for the determination of the upright direction can be ensured increases, and the number of cases where the determination result becomes indeterminate can be reduced. As a result, the accuracy of the process of identifying the upright direction can be improved.

[0074] <Fourth Aspect> In the image processing apparatus of this aspect, the counting unit 70c counts the number of adjacent character images among the character images whose actual size exceeds the upper limit value in the rectangular regions within a row or a column among the rectangular regions divided by the region division unit 70b, and When the number of adjacent character images counted by the counting unit 70c becomes two or more for the first time within a row or a column, the control unit 70 controls so that the upper limit value of the size becomes larger. According to this aspect, in step S505, when both of the two adjacent rectangular regions are excluded, since the actual character size in the image may be larger than expected, by setting a larger upper limit value for the size of the rectangular region, the number of rectangular regions sufficient to identify the upright direction can be ensured in more cases, and the cases where the upright direction is indeterminate can be reduced. As a result, the accuracy of the process of identifying the upright direction can be improved.

[0075] <Aspect 5> The control unit 70 of this aspect is characterized in that when the number of character images counted by the counting unit 70c is more than half within the same row or the same column, it controls so that the upper limit value of the size becomes larger. According to this aspect, when more than half of the rectangular regions are excluded, since the actual character size in the image is likely to be larger than expected, by setting a larger upper limit value for the size of the rectangular region, the number of rectangular regions sufficient for the process of identifying the upright direction can be ensured even more, and the cases where the upright direction is indeterminate can be reduced. As a result, the accuracy of the process of identifying the upright direction can be improved.

[0076] <Aspect 6> The control unit 70 of this aspect extracts a character image whose actual size exceeds the upper limit value of the size, and when the extracted character image straddles adjacent rows or columns, it controls to inherit the upper limit value of the size. According to this aspect, when the character image straddles adjacent rows or columns, by controlling to inherit the upper limit value of the size, the upper limit value can be updated efficiently.

[0077] <Aspect 7> In the image processing apparatus according to this aspect, the confidence calculation unit 70d performs character recognition processing along at least two directions out of the + direction and the - direction in the vertical direction, and the + direction and the - direction in the horizontal direction for the character image within the rectangular area, calculates the confidence levels in at least two directions, The upright direction determination unit 70e determines, as the upright direction, the direction in which each of the confidence levels in at least two directions calculated by the confidence calculation unit 70d has a confidence level greater than a predetermined threshold value. The control unit 70 is characterized in that when controlling to increase the upper limit value of the size, it controls to increase the threshold value of the confidence level. According to this aspect, since the rectangular area of a character larger than the originally assumed size is targeted for voting, by increasing the threshold value of the confidence level, it is possible to avoid the drawback that a rectangle that is actually a photo rather than a character (a rectangle that does not look like a character = a rectangle with a low confidence level) becomes the target of voting. As a result, in a state where the drawback is reduced, the upper limit value of the rectangular area of the character can be set large, and as a result, the accuracy of the upright direction determination process can be improved.

[0078] <Eighth Aspect> The upright direction determination unit 70e of this aspect integrates the confidence level only for the direction in which the confidence level is the maximum when at least one of the confidence levels in the four directions calculated by the confidence calculation unit 70d exceeds the first threshold value, and when the value obtained by dividing the integrated confidence level for each direction by the number of integration times exceeds a second threshold value different from the first threshold value, determines the direction as the upright direction. According to this aspect, the confidence level regarding a character is not reliable enough with only one character. On the other hand, since an extremely high confidence level can be trusted, by setting the threshold value (vertical and horizontal discrimination threshold value) for the value obtained by normalizing the confidence levels of multiple characters to be larger than the confidence level threshold value for one character, the accuracy of the upright direction determination process can be ensured.

[0079] <Ninth Aspect> When the upright direction determination unit 70e of the control unit 70 of this aspect determines the upright direction, the control unit 70 controls so that the upper limit value of the size set in the region division unit 70b becomes larger than when performing normal character recognition processing, and based on the increased upper limit value, controls to determine the upper limit value of the character size in normal character recognition processing. According to this aspect, since the upper limit value of the size of the rectangular area of the character reviewed during the upright direction determination process is also used during the subsequent OCR process, if the upright direction determination process does not become indeterminate, the reviewed rectangular size is likely to be appropriate, and characters that are (erroneously) excluded during the subsequent OCR process can be reduced. As a result, the recognition rate of the OCR process can be improved.

[0080] <Aspect 10> The image processing method of this aspect is an image processing method for identifying the upright direction of character images included in image data and performing normal character recognition processing along the upright direction. For each character image included in the image data, an upper limit value is set for the size in the vertical direction and the horizontal direction, and the region division step of dividing into rectangular areas within the upper limit value, the confidence calculation step of performing character recognition processing along the + direction and - direction in the vertical direction, and the + direction and - direction in the horizontal direction for the character image within the rectangular area and calculating the confidence of 4 directions, and the upright direction determination step of determining the direction having the maximum confidence among the confidences of the 4 directions calculated by the confidence calculation step as the upright direction, and a control step are executed. The control step is characterized in that when the upright direction determination step determines the upright direction, it controls so that the upper limit value of the size set in the region division step becomes larger than when performing normal character recognition processing. According to this aspect, since the upper limit value of the size set in the region division step can be controlled to be larger, a rectangular area of a character larger than the original assumption can be the target of voting, so the case where the determination of the upright direction becomes indeterminate can be reduced, and as a result, the accuracy of the process of identifying the upright direction can be improved.

[0081] <Aspect 11> The program of this aspect is a program for causing a processor to function as an image processing method for identifying the upright direction of character images included in image data and performing normal character recognition processing along the upright direction. The program causes the processor to perform an area division step of setting upper limit values for the vertical and horizontal sizes for each character image included in the image data and dividing the image data into rectangular areas within the upper limit values, a confidence calculation step of performing character recognition processing along the + direction and - direction in the vertical direction and the + direction and - direction in the horizontal direction for the character images within the rectangular areas and calculating the confidence levels in four directions, an upright direction determination step of determining, as the upright direction, the direction having the maximum confidence level among the confidence levels in the four directions calculated in the confidence calculation step, and a control step. The control step functions to control such that, when the upright direction determination step determines the upright direction, the upper limit value of the size set in the area division step is larger than when performing normal character recognition processing. According to this aspect, since it is possible to control such that the upper limit value of the size set in the area division step becomes larger, it is possible to target rectangular areas of characters larger than originally assumed for voting, so that cases where the determination of the upright direction becomes indefinite can be reduced, and as a result, the accuracy of the process for identifying the upright direction can be improved.

Explanation of Signs

[0082] 1…System, 10…Image forming apparatus, 101…CPU, 102…ROM, 103…RAM, 104…HD, 105…HDD controller, 106…Display, 111…Keyboard, 112…Pointing device, 114…RW drive, 115…Recording medium, 20…Controller, 22a…ROM, 22b…RAM, 25a…CPU, 25b…NB, 26…ASIC, 28…HDD controller, 29…HD, 30…Engine control unit, 31…Scanner unit, 32…Printer unit, 40…Operation panel, 40a…Panel display unit, 40b…Operation panel, 60…Short-range communication circuit, 60a…Communication circuit, 70…Control unit, 70a…Binary image input unit, 70b…Area division unit, 70c…Counting unit, 70d…Confidence calculation unit, 70e…Upright direction determination unit, 80…PC, 90…PC

Prior Art Documents

Patent Documents

[0083]

Patent Document 1

Claims

Claim 1 An image processing apparatus that identifies the upright direction of character images included in image data and performs normal character recognition processing along the upright direction, comprising: a region division unit that sets upper limit values for the sizes in the vertical direction and the horizontal direction for each character image included in the image data, and divides the image data into rectangular regions within the upper limit values; a confidence calculation unit that performs character recognition processing along the + direction and the - direction in the vertical direction, and the + direction and the - direction in the horizontal direction for the character images within the rectangular regions, respectively, and calculates the confidence levels in the four directions; an upright direction determination unit that determines, as the upright direction, the direction having the maximum confidence level among the confidence levels in the four directions calculated by the confidence calculation unit; a control unit; and when the upright direction determination unit determines the upright direction, the control unit controls the upper limit value of the size set for the region division unit to be larger than when performing the normal character recognition processing. The image processing apparatus is characterized by this. Claim 2 When the actual value of the size does not exceed the upper limit value of the size, and when the determination result by the upright direction determination unit becomes indeterminate, the control unit controls the upper limit value of the size to be larger. The image processing apparatus according to claim 1 is characterized by this. Claim 3 The control unit extracts a character image whose actual value of the size exceeds the upper limit value of the size, and when the extracted character image straddles adjacent rows or columns, controls to inherit the upper limit value of the size. The image processing apparatus according to claim 1 is characterized by this. Claim 4 The confidence calculation unit performs character recognition processing along at least two of the + direction and the - direction in the vertical direction, and the + direction and the - direction in the horizontal direction for the character images within the rectangular regions, respectively, and calculates the confidence levels in at least the two directions; the upright direction determination unit determines, as the upright direction, the direction having a confidence level greater than a predetermined threshold value among the confidence levels in at least the two directions calculated by the confidence calculation unit; when the control unit controls the upper limit value of the size to be larger, the control unit controls the threshold value of the confidence level to be larger. The image processing apparatus according to any one of claims 1 to 3 is characterized by this. Claim 5 When at least one of the confidence levels in the four directions calculated by the upright direction determination unit exceeds a first threshold value, the confidence levels are integrated only for the direction with the maximum confidence level, and when the value obtained by dividing the integrated confidence level for each direction by the number of integrations exceeds a second threshold value different from the first threshold value, the image processing apparatus according to claim 4, characterized in that the direction is set as the upright direction.

6. When the upright direction determination unit determines the upright direction, the control unit controls so that the upper limit value of the size set in the region division unit becomes larger than when performing the normal character recognition process, and based on the increased upper limit value, the control unit controls to determine the upper limit value of the character size in the normal character recognition process. The image processing apparatus according to any one of claims 1 to 5, characterized in that it is performed.

7. An image processing method for identifying the upright direction of a character image included in image data and performing normal character recognition processing along the upright direction, A region division step of setting an upper limit value for the vertical and horizontal sizes for each character image included in the image data and dividing the image into rectangular regions within the upper limit value; A confidence level calculation step of performing character recognition processing along the + direction and - direction in the vertical direction, and the + direction and - direction in the horizontal direction for the character image within the rectangular region, and calculating the confidence levels in the four directions; An upright direction determination step of determining, as the upright direction, the direction having the maximum confidence level among the confidence levels in the four directions calculated by the confidence level calculation step; A control step, and The control step is characterized in that, when the upright direction determination step determines the upright direction, it controls so that the upper limit value of the size set in the region division step becomes larger than when performing the normal character recognition process. Image processing method.

8. A program for causing a processor to function as an image processing method for identifying the upright direction of a character image included in image data and performing normal character recognition processing along the upright direction, The processor is A region division step of setting an upper limit value for the vertical and horizontal sizes for each character image included in the image data and dividing the image into rectangular regions within the upper limit value; A confidence calculation step of performing character recognition processing along the + direction and the - direction in the vertical direction, and the + direction and the - direction in the horizontal direction for the character image within the rectangular area, and calculating the confidence levels in the four directions; An upright direction determination step of determining, as the upright direction, the direction having the maximum confidence level among the confidence levels in the four directions calculated by the confidence calculation step; A control step, and is provided with; The control step is a program for functioning to control such that when the upright direction determination step makes a determination of the upright direction, the upper limit value of the size set in the region division step becomes larger than when performing the normal character recognition processing.

Images