Image processing device, image processing method, and program

Abstract

To provide an image processing system capable of swiftly identifying appropriate information by executing character recognition only in a required part of the document image.SOLUTION: Character areas in an image are detected, and the priority related to character recognition processing is set for each character area. Time factor values related to the processing time of character recognition processing for each character area are set based on information related to the size of the pixel block in the character area, and based on the set priority and time factor values, the character areas and the execution order to execute character recognition processing are determined.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] There are systems that use the results of character recognition of character information from document images for image indexing. One indexing method is to use the title of the document image as the file name of the image. There are also systems that identify areas in the document image where names, amounts, etc. are written, and extract information by performing character recognition. In these systems, the location of the character information to be extracted from the document image varies depending on the user's request, and it is not necessarily necessary to perform character recognition on all character areas of the input document image.

[0003] Here, there is a demand for character recognition of only necessary parts of a document image within a predetermined time, taking into account the responsiveness to a user and the resources of a processing device. Patent Document 1 proposes a technique for setting a time limit for character recognition for each text block in a document image based on the area, and controlling the execution order based on the number of characters and character size obtained by performing preliminary character recognition. Patent Document 1 also describes that when the character recognition process of a text block reaches a previously set time limit, the process transitions to character recognition process of the next text block. Patent Document 2 proposes a technique for analyzing attributes such as character size and character color for each area of ​​a document image, performing character recognition from an area of ​​large characters of a predetermined color, and terminating the character recognition process when a predetermined number of characters is reached. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2013-161268 A [Patent Document 2] JP 2012-178007 A Summary of the Invention [Problem to be solved by the invention]

[0005] In Patent Document 1 and Patent Document 2, the execution order and processing time of character recognition are set based on the position of the area in the document image, the character size, and the character color. However, in order to recognize a low-quality character image, specifically a character that is faint or in contact with surrounding pixels, more processing is required compared to an ideal character image. In other words, in character recognition processing, there are factors that increase the processing time that cannot be determined by the coordinate position, size, and color of the character area, and character recognition processing of the necessary area must be performed while preventing these unexpected increases in processing time. Furthermore, in Patent Document 1, there is a risk that the result of character recognition may vary due to changes in the time required for actual character recognition processing due to system load. An object of the present invention is to perform character recognition only on necessary portions of a document image, thereby enabling appropriate information to be identified quickly. [Means for solving the problem]

[0006] The image processing device according to the present invention includes a detection unit for detecting a character area in an image, and a , classifying pixel blocks in the character region based on size, and based on the number of pixel blocks belonging to each of the classified sizes, The system is characterized by having a priority setting means for setting a priority related to character recognition processing, a time factor value setting means for setting a time factor value related to the processing time of character recognition processing for each character area based on information related to the size of pixel chunks within the character area, and an execution order setting means for determining the character areas for which character recognition processing is to be performed and the order of execution based on the set priority and time factor value. Effect of the Invention

[0007] According to the present invention, character recognition can be performed only on necessary portions of a document image, making it possible to quickly identify appropriate information. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of the configuration of an image processing system according to a first embodiment. [Diagram 2] 5 is a flowchart showing an example of character recognition processing according to the first embodiment. [Figure 3A] FIG. 2 is a diagram showing an example of a scanned document image. [Figure 3B] FIG. 11 is a diagram showing an example of a detection result of a character region. [Figure 4] 10 is a flowchart illustrating an example of a recognition priority setting process according to the first embodiment. [Diagram 5] 13 is a flowchart illustrating an example of a time factor value setting process according to the first embodiment. [Figure 6] 5A to 5C are diagrams illustrating an example of intermediate results of a character recognition execution order according to the first embodiment. [Figure 7] FIG. 11 is a diagram showing an example of a character recognition execution order. [Figure 8] 13 is a flowchart showing an example of a character recognition process and a time factor value setting process according to a second embodiment. [Figure 9] FIG. 11 is a diagram showing an example of a character region according to the second embodiment. [Figure 10] FIG. 11 is a diagram showing another example of a character region according to the second embodiment. [Figure 11] 13 is a flowchart showing an example of character recognition processing and direction determination processing according to the third embodiment. [Figure 12] FIG. 13 is a diagram illustrating an execution range of character recognition according to the third embodiment. [Figure 13] 13A to 13C are diagrams illustrating examples of execution images for direction determination according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] (First embodiment) A first embodiment of the present invention will be described below. Fig. 1 is a block diagram showing an example of the configuration of an image processing system according to the first embodiment. The image processing system in this embodiment includes a reading device 100 and an image processing device 110.

[0011] The reading device 100 includes a scanner unit 101 and a communication unit 102. The scanner unit 101 reads a document and generates a scanned document image. The communication unit 102 communicates with an external device via a network. The communication unit 102 transmits, for example, the document image generated by the scanner unit 101 to the image processing device 110.

[0012] The image processing device 110 includes a system control unit 111, a ROM 112, a RAM 113, a hard disk drive (HDD) 114, a display unit 115, an input unit 116, and a communication unit 117. The system control unit 111 reads out a control program stored in a ROM (Read Only Memory) 112 and executes various processes. The RAM (Random Access Memory) 113 is used as a temporary storage area such as a main memory or work area of ​​the system control unit 111. The HDD 114 stores various data, various programs, and the like. Note that the functions and processes of the image processing device 110 described below are realized by the system control unit 111 reading out a program stored in the ROM 112 or the HDD 114 and executing the program. The system control unit 111 reads out a program from the ROM 112 or the like and executes the program, thereby realizing functions such as a detection means, a priority setting means, a time factor value setting means, and an execution order setting means.

[0013] The display unit 115 displays various information. The input unit 116 has a keyboard and a mouse, and accepts various operations by the user. The communication unit 117 performs communication processing with an external device via a network. The communication unit 117 receives a document image from the reading device 100, for example. The display unit 115 and the input unit 116 may be provided integrally, such as a touch panel. The display unit 115 may also be a unit that projects an image using a projector, and the input unit 116 may be a unit that recognizes the position of a fingertip relative to the projected image using a camera or the like.

[0014] In this embodiment, a scanner unit 101 of the reading device 100 reads a paper document such as a form and generates a scanned document image. The scanned document image is transmitted to an image processing device 110 by a communication unit 102 of the reading device 100. In the image processing device 110, a communication unit 117 receives the scanned document image from the reading device 100 and stores the received document image in a storage device such as an HDD 114. Note that some functions of the display unit 115 and the input unit 116 may be provided in the reading device 100.

[0015] Fig. 2 is a flowchart showing an example of character recognition processing in the image processing device 110 in this embodiment. The processing of the flowchart shown in Fig. 2 is realized by the system control unit 111 executing the program stored in the ROM 112 in the image processing device 110.

[0016] In S200, the reading device 100 scans a paper document using the scanner unit 101 in accordance with an instruction from a user, and transmits the scanned document image to the image processing device 110 via the communication unit 102. The transmitted scanned document image is received by the communication unit 117 of the image processing device 110, and stored in a storage unit such as the HDD 114 by the system control unit 111. Fig. 3A shows an example of an image (scanned document image) obtained by the process in S200.

[0017] In S201, the system control unit 111 performs character area detection processing on the scanned document image and stores the detection result in the RAM 113. The character area detection processing is processing for detecting an area (character area) including characters to be recognized from the scanned document image. The method for detecting the character area is not particularly limited, and known techniques can be used. An example of the character area detection technique is the processing described in U.S. Pat. No. 5,680,478. In the above example, pixel blocks and white pixel block clusters in the document image are extracted, and characteristic areas such as characters, pictures, figures, tables, frames, and lines are extracted based on their shapes, sizes, clustering states, and the like. FIG. 3B is a diagram showing an example of the character area detection result obtained by the processing in S201. In the example shown in FIG. 3B, character areas 301 to 325 are detected in the scanned document image 300.

[0018] In S202, the system control unit 111 performs a recognition priority setting process for each character region included in the scanned document image, and stores the result in the RAM 113. Here, the recognition priority is a parameter value that ranks whether the character region is suitable for the purpose of character recognition. In the example described below, the recognition priority is set to 1 as the highest and 5 as the lowest. A character region with many pixel chunks that are assumed to be noise or small point characters has a low priority for character recognition. On the other hand, in the case of a slip image such as that shown in FIG. 3A and FIG. 3B, a character region composed of pixel chunks larger than the normal size such as the amount or date should be preferentially subjected to character recognition processing. Therefore, in this embodiment, a recognition priority indicating the priority for performing character recognition processing is determined based on the distribution tendency of the size of the pixel chunks in the character region obtained by the character region detection process in S201.

[0019] FIG. 4 is a flowchart showing an example of the recognition priority setting process in this embodiment, which is executed in S202 shown in FIG. In S400, the system control unit 111 refers to the RAM 113 and acquires the character area to be processed.

[0020] Next, in S401, the system control unit 111 classifies and tallies the pixel blocks extracted from the character area acquired in S400 by size, and stores the results in the RAM 113. Here, the pixel blocks are classified into three patterns, noise size, small point size, and main character size, based on the height of the pixel blocks. For example, in the case of extracting the amount and date from a slip image, the threshold value for classifying the pixel blocks is the following value obtained by converting the expected number of character points into the number of pixels, and classification is performed as shown in the following conditional formula (1). Noise Size Hi <Tq_1 Small point size Tq_1≦Hi <Tq_2 …(1) Main text size Tq_2≦Hi In conditional expression (1), Hi is the height of the pixel block, Tq_1 is the number of pixels corresponding to a character size of 3pt (point), and Tq_2 is the number of pixels corresponding to a character size of 6pt.

[0021] Then, the system control unit 111 stores the number of pixel blocks in the character area classified according to conditional formula (1) in the following variables for each classification, and stores the results in the RAM 113. The number of pixel blocks of noise size is stored in the variable Cnt_noise, the number of pixel blocks of small point size is stored in the variable Cnt_small, and the number of pixel blocks of main character size is stored in the variable Cnt_normal.

[0022] Next, in S402, the system control unit 111 refers to the RAM 113 and determines whether or not a pixel block classified as the main character size exists in the character area to be processed. That is, the system control unit 111 determines whether or not Cnt_noise>0. If it is determined that a pixel block classified as the main character size exists (Yes), the system control unit 111 transitions to S403. On the other hand, if it is determined that a pixel block classified as the main character size does not exist (No), the system control unit 111 transitions to S405.

[0023] In S403, the system control unit 111 refers to the RAM 113 and determines whether the number of pixel blocks classified as the main character size in the processing target character area is greater than the total number of pixel blocks classified as the noise size and the small point size. In other words, the system control unit 111 determines whether Cnt_normal>(Cnt_small+Cnt_noise).

[0024] If it is determined that the number of pixel blocks classified as the main character size is greater than the total number of pixel blocks classified as the noise size and the small point size (Yes in S403), the system control unit 111 transitions to S406. In S406, the system control unit 111 sets the recognition priority of the character region to be processed to 1, stores it in the RAM 113, and ends the recognition priority setting process.

[0025] On the other hand, if it is determined that the number of pixel blocks classified as the main character size is not greater than the total number of pixel blocks classified as noise size and small point size (No in S403), the system control unit 111 transitions to S404. In S404, the system control unit 111 refers to the RAM 113 and determines whether or not the total number of pixel blocks classified as noise size and small point size in the character area to be processed is greater than a predetermined threshold value Th. In other words, the system control unit 111 determines whether or not (Cnt_small+Cnt_noise)>Th.

[0026] If it is determined that the total number of pixel blocks classified as noise size and small point size is greater than the predetermined threshold Th (Yes in S404), the system control unit 111 transitions to S407. In S407, the system control unit 111 sets the recognition priority of the character region to be processed to 2, stores this in the RAM 113, and ends the recognition priority setting process.

[0027] On the other hand, if it is determined that the total number of pixel chunks classified as noise size and small point size is not greater than the predetermined threshold Th (No in S404), the system control unit 111 transitions to S408. In S408, the system control unit 111 sets the recognition priority of the processing target character region to 3, stores this in the RAM 113, and ends the recognition priority setting process.

[0028] In S405, the system control unit 111 refers to the RAM 113 and determines whether the number of pixel blocks classified as small point size is greater than the number of pixel blocks classified as noise size in the character area to be processed. In other words, the system control unit 111 determines whether Cnt_small>Cnt_noise.

[0029] If it is determined that the number of pixel blocks classified as small point size is greater than the number of pixel blocks classified as noise size (Yes in S405), the system control unit 111 transitions to S409. In S409, the system control unit 111 sets the recognition priority of the character region to be processed to 4, stores this in the RAM 113, and ends the recognition priority setting process.

[0030] On the other hand, if it is determined that the number of pixel blocks classified as small point size is not greater than the number of pixel blocks classified as noise size (No in S405), the system control unit 111 transitions to S410. In S410, the system control unit 111 sets the recognition priority of the character region to be processed to 5, stores it in the RAM 113, and ends the recognition priority setting process.

[0031] The above is the recognition priority setting process in S202. If the recognition priority is low, the character area will have many normal-sized pixel blocks and few small-point characters and noise, i.e., the area will be suitable for character recognition purposes, and the recognition priority will be set high. Note that in this embodiment, the recognition priority is determined using only the height of the pixel block, but the recognition priority may also be determined using other features such as coordinate information of the character area. For example, if a character area is located at an upper position on the image plane, it is highly likely to be the title of a document, so a process of increasing the recognition priority may be added.

[0032] Returning to FIG. 2, next in S203, the system control unit 111 performs a time factor value setting process for each character area included in the scanned document image, and stores the results in the RAM 113. Here, the time factor value is a value that estimates the length of processing time when character recognition is performed on the character area to be processed, and the larger the value, the longer the processing time required. The time factor value basically depends on the number of characters included in the character area. Here, a virtual number of characters is estimated from information only on pixel blocks that can be obtained from the image without performing character recognition, and the above-mentioned time factor value is calculated from the estimated virtual number of characters.

[0033] In addition, more processing is required to accurately recognize character regions that are difficult to read, such as those where characters are blurred, faint, or touching underlines, and the like, and processing time is expected to increase. Furthermore, symbols such as periods and midpoints, and hyphens and minus signs often have similar shapes. Also, dotted lines and long horizontal bars require more attempts to split into single characters in the character recognition process. For this reason, the processing time required to output the final character recognition results is often longer. Therefore, the system control unit 111 classifies whether the pixel blocks are difficult to read based on their arrangement and shape, and sets a coefficient that becomes a penalty depending on the classification. In the following explanation, the classification of whether the pixel blocks are difficult to read will be called the difficulty level.

[0034] That is, in this embodiment, the calculation formula for the time factor value Tf is defined as follows. Tf = Σ(Ci × Ni) … (2) In formula (2), Ni is the virtual number of characters at the difficulty level i, and Ci is the coefficient of the difficulty level i. The difficulty level coefficient Ci is defined as follows: Ci = αi × βi … (3) In equation (3), αi is the size factor coefficient of the difficulty level i, and βi is the contact factor coefficient of the difficulty level i.

[0035] In this embodiment, the level of difficulty is divided into five patterns, namely, noise, small point only, main size only, small point touching, and main size touching, using only information obtained from pixel blocks, and the time factor value is calculated by setting the virtual number of characters and the difficulty level coefficient. Figure 5(A) is a flowchart showing an example of the time factor value setting process executed using the above-mentioned formula (2).

[0036] In S500, system control unit 111 acquires a character region to be processed by referring to RAM 113. System control unit 111 executes the processes of S501 to S513 described below for all pixel blocks included in the character region to be processed.

[0037] Next, in S501, the system control unit 111 obtains the height and width of one pixel block from the character area, and stores the height and width in the RAM 113. Since the height of the pixel block was previously calculated in S401, the information may be obtained from the RAM 113.

[0038] Next, in S502, the system control unit 111 determines a size factor coefficient of the pixel block based on the height information of the pixel block with reference to the RAM 113. If the system control unit 111 classifies the pixel block as noise, the process proceeds to S503, if the system control unit 111 classifies the pixel block as a small point character, the process proceeds to S504, and if the system control unit 111 classifies the pixel block as a main size character, the process proceeds to S505.

[0039] In S503, the system control unit 111 sets the size factor coefficient of the pixel block classified as noise to α0, and stores the result in the RAM 113. Then, the process proceeds to S509. In S504, the system control unit 111 sets the size factor coefficient of the pixel block classified as a small point character to α1, and stores the result in the RAM 113. Then, the process proceeds to S506. In S505, the system control unit 111 sets the size factor coefficient of the pixel block classified as a main size character to α2, and stores the result in the RAM 113. Then, the process proceeds to S506.

[0040] In S506, the system control unit 111 refers to the RAM 113, determines the touching factor coefficient of the pixel block based on the width information of the pixel block, and stores the result in the RAM 113. If the system control unit 111 classifies the pixel block as a single character, the process proceeds to S507, and if the system control unit 111 classifies the pixel block as a touching character, the process proceeds to S508. In S507, the system control unit 111 sets the contact factor coefficient of the pixel block classified as a single character to β1, and stores the result in the RAM 113. Then, the process proceeds to S510. In S508, the system control unit 111 sets the touching factor coefficient of the pixel block classified as a touching character to β2, and stores the result in the RAM 113. Then, the process proceeds to S511.

[0041] In S509, the system control unit 111 sets the virtual number N of characters for the pixel block whose readability level has become noise to 1, stores the result in the RAM 113, and transitions to S512. In S510, the system control unit 111 sets the virtual number N of characters for the pixel whose readability level has become a single character to 1, stores the result in the RAM 113, and transitions to S512. In S511, the system control unit 111 sets the virtual number of characters N for the pixel block whose readability level is a touching character to n, stores the result in the RAM 113, and transitions to S512. Since the number of characters is merely estimated here, n is set to a value obtained by dividing the width of the pixel block by a prescribed threshold. The prescribed threshold may be, for example, the number of pixels in the height of the pixel block, or a fixed value may be set. In other words, at least the virtual number of characters N is set to a value of 2 or more.

[0042] In S512, the system control unit 111 calculates the coefficient Ci of the level of difficulty in reading according to the above-mentioned formula (3). That is, the system control unit 111 sets the coefficient Ci of the level of difficulty in reading for the pixel block to a value obtained by multiplying the coefficients α and β calculated earlier, stores the result in the RAM 113, and transitions to S513.

[0043] In S513, the system control unit 111 obtains the obscurity level coefficient C and the number of characters N of the pixel block from the RAM 113, and adds the multiplied value to the time factor value of the character region being processed. The process in S513 corresponds to the above-mentioned formula (2). That is, the system control unit 111 calculates the obscurity level coefficient C and the virtual number of characters N for all pixel blocks in the character region, sets the sum as the time factor value Tf of the character region, and stores it in the RAM 113.

[0044] FIG. 5B shows a list of coefficients C and virtual numbers of characters N for each level of difficulty set in the above-mentioned recognition priority setting process. In this embodiment, based on the height information of pixel blocks, the character recognition process is simply classified into noise, which is an area where character recognition processing is not required, small point characters where the characters are blurred and difficult to analyze, and main size characters. The coefficients set in each process are small for main size characters and single characters, and large for noise, small point characters, and touching characters. The threshold value of the criterion for determining whether character recognition requires processing time is set to the same as that of the recognition priority setting process in consideration of the calculation cost of the determination process, but the determination threshold value may be changed depending on the target image group and the classification of the difficult-to-read state.

[0045] In addition, since the small point character of the lowest limit to be recognized varies depending on the characteristics of the character recognition algorithm that actually executes the character recognition and the target language, a judgment threshold value may be set according to the character recognition processing process actually used. Similarly, the characters that are difficult to classify vary depending on the character recognition algorithm. Therefore, a level of difficulty in reading may be prepared separately according to the characteristics of the algorithm. For example, if a character recognition algorithm has a processing time for italic characters, a case in which the character is classified as an italic character based on the tendency of the shape of the pixel block may also be set. Furthermore, the coefficient may be made different depending on the characteristics of the character recognition algorithm and the target language to be recognized. For example, character recognition of East Asian languages ​​that use Chinese characters such as Japanese and Chinese has a higher processing cost than character recognition of European and American languages. Therefore, if the character area is an East Asian language, the coefficient is set to a large value. The above is the time factor value setting process in S203.

[0046] 2, next, in S204, the system control unit 111 sets the execution order of character recognition for character areas in the acquired document image based on the recognition priority set in S202 and the time factor value set in S203, and stores the result in the RAM 113. Then, the system control unit 111 proceeds to S205.

[0047] 6(A) shows an example (intermediate result 600) in which the processes of S202 and S203 shown in FIG. 2 are performed on character areas 301-325 detected in scanned document image 300, and recognition priorities and time factor values ​​are calculated. The order of character recognition is first sorted by highest recognition priority (lowest numerical value), and character areas with the same priority are sorted by lowest time factor value (lowest numerical value). This results in an order of document image suitable for the purpose of character recognition and requiring the least amount of time to execute character recognition. FIG. 6(B) shows an example (intermediate result 601) in which intermediate result 600 is sorted as described above.

[0048] Next, in S205, the system control unit 111 calculates the cumulative total of time factor values ​​from the top of the execution order of character recognition set in S204, and sets a character area (character recognition execution area) for performing character recognition based on the cumulative total of time factor values. The system control unit 111 stores the result of setting the execution area of ​​character recognition within a range in which the sum of time factor values ​​is lower than a preset threshold in the RAM 113, and transitions to S206. Here, the threshold for the sum of time factor values ​​can be a fixed value that is predetermined based on the CPU speed and required response time of the system. FIG. 6C shows an example (intermediate result 602) in which the cumulative total of time factor values ​​is calculated from the intermediate result 601. For example, if the threshold that is the upper limit of the cumulative total of the preset time factor values ​​is set to 250, the character area for which character recognition is actually performed is the character area in range 603 when expressed in a list, and the character area shown in 604 in FIG. 7 when expressed in a document image.

[0049] In S206, the system control unit 111 executes character recognition according to the order and range of the character areas set in the processes in S204 and S205, and stores the result in a storage unit such as the RAM 113 or the HDD 114.

[0050] As described above, character regions in a document image can be processed in an order that is suitable for the purpose of character recognition and requires the least amount of processing time. In addition, since the real time of the character recognition process is not used to limit the area to be processed, the results of character recognition are not affected by the system load. This makes it possible to extract text information from character regions required by the user while maintaining responsiveness, and to quickly identify appropriate information from required character regions in a document image.

[0051] Second Embodiment Next, a second embodiment of the present invention will be described. In the first embodiment, the execution order of character recognition is set in units of the character area extracted first. On the other hand, there are cases where the character to be recognized and noise or symbols with a high time factor value are mixed in the same extracted character area. Therefore, when a character area with a high recognition priority but a high time factor value is present in a document image, a pixel block that is a factor that particularly pushes up the time factor value may be identified, and the pixel block may be deleted, and the time factor value may be recalculated and character recognition may be performed. Explanations of contents common to the first embodiment, such as the configuration and control flow, will be omitted, and a time factor value resetting process (S800 shown in FIG. 8A) that differs from the first embodiment will be described. The time factor value resetting process is performed after the time factor value setting process is performed in S203.

[0052] FIG. 8B is a flowchart showing an example of the time factor value resetting process in this embodiment. In S801, the system control unit 111 refers to the RAM 113 and acquires the character areas for which the time factor values ​​have not been recalculated and the cumulative value of the time factor values ​​within the character areas. In S802, the system control unit 111 determines whether the cumulative value of the acquired time factor values ​​is greater than a predetermined threshold. If it is determined that the cumulative value of the acquired time factor values ​​is greater than the predetermined threshold (Yes in S802), the system control unit 111 transitions to S803. On the other hand, if it is determined that the cumulative value of the acquired time factor values ​​is not greater than the predetermined threshold (No in S802), the system control unit 111 transitions to S801.

[0053] In S803, the system control unit 111 refers to the RAM 113, identifies a pixel block (factor pixel block) that is a factor of increasing the time factor value of the character region, and stores the result in the RAM 113. Specifically, a pixel block to which a large coefficient used in the calculation of the time factor value is assigned in the process of S513 in the flowchart shown in Fig. 5 is identified. In this embodiment, among the pixel blocks in the character region, a pixel block to which a noise coefficient is assigned is determined to be a factor pixel block.

[0054] In S804, the system control unit 111 removes the identified factor pixel block from the character region. For example, when the character region 325 shown in Fig. 3B is input, as shown in Fig. 9, the pixel blocks to which the noise coefficients included in the character region 325 shown in Fig. 9(A) are assigned are removed by this process, and the character region 325_1 shown in Fig. 9(B) is output. Then, in S805, the system control unit 111 performs a time factor value setting process for the character region from which the factor pixel block has been deleted, in the same manner as in S203 shown in Fig. 2.

[0055] As a result, pixel blocks with large time factor values ​​contained in a character area can be excluded and the order of character recognition execution can be changed to a higher position, making it possible to perform character recognition on characters that the user expects to be useful, even within the same character area.

[0056] In the above example, pixel clusters with high time factor values ​​in the character region are removed and the time factor values ​​are recalculated. The pixel clusters with high time factor values ​​may be set as new separated character regions, new recognition priorities and time factor values ​​may be calculated, and the character recognition execution order may be set again. For example, character region 325 shown in FIG. 10(A) may be separated into two character regions, character regions 325_2 and 325_3, as shown in FIG. 10(B), recognition priorities and time factor values ​​may be calculated, and the character recognition execution order may be set again. This makes it possible to recognize the upper region in a scene that takes responsiveness into consideration, and perform character recognition on the other regions in a follow-up manner, and ultimately perform character recognition on all character regions in a character image.

[0057] In the above-described embodiments, the recognition priority is calculated for all character areas in a scanned document image, and a time factor value is calculated for all character areas. However, once the recognition priority of all character areas has been calculated, the character areas are rearranged. The time factor value may then be calculated in descending order of recognition priority according to the processing order of the character areas, and the processing may be terminated when the time factor value exceeds a default value, thereby determining the areas to be subjected to character recognition and the execution order. In this case, the time factor value setting process for character area images with even lower recognition priority may be omitted, thereby enabling further improvement in responsiveness.

[0058] (Third embodiment) Next, a third embodiment of the present invention will be described. In the first and second embodiments described above, the character area to be processed and its execution order are defined, and character recognition is performed for each character area in sequence according to the determined execution order. Meanwhile, the document image acquired from the scanner unit 101 may be rotated in 90 degree increments. For this reason, before executing the character recognition process, it is necessary to execute a direction determination process for determining the direction in which the characters in the document image are erected. In the direction determination, character recognition is generally performed for each of the images internally rotated in four different directions by 90 degrees, and the direction with the highest reliability of the character recognition result is determined to be the erect direction of the document.

[0059] Therefore, for direction determination as well, a character area to be processed may be set based on the recognition priority and the time factor value. Descriptions of contents common to the first embodiment, such as the configuration and control flow, will be omitted. In the third embodiment, the system control unit 111 reads out and executes a program from the ROM 112 or the like, thereby realizing functions such as a detection means, a priority setting means, a time factor value setting means, an execution area setting means, an image generation means, and a direction determination means. In the following, the direction determination process and image rotation process (S1101 and S1102 shown in FIG. 11(A)) in the third embodiment that differ from the first embodiment will be described.

[0060] 11A is a flowchart showing an example of character recognition processing in the image processing device 110 according to the third embodiment. The direction determination processing in S1101 is executed after acquiring the scanned document image in S200.

[0061] In S1101, the system control unit 111 determines the direction in which characters are upright for the scanned document image, and stores the angle of the upright character direction in the RAM 113. Here, the character direction is defined as the direction of characters in the document when the upright character direction is 0 degrees, and in S1101, the character direction in which characters in the document are upright is output in 90-degree increments (0 degrees, 90 degrees, 180 degrees, 270 degrees).

[0062] In S1102, the system control unit 111 refers to the RAM 113, acquires the angle of the upright character direction, and rotates the scanned document image based on the acquired angle of the character direction. This results in a document image with upright characters. After that, the system control unit 111 transitions to S201 (character area detection process).

[0063] FIG. 11(B) is a flowchart showing an example of a direction determination process in the third embodiment. In the flowchart shown in FIG. 11(B), the process related to setting the recognition priority and the time factor value that is common to the first embodiment will not be described.

[0064] In S205, the system control unit 111 sets a character area (character recognition execution area) for performing character recognition within a range where the cumulative total of the time factor values ​​is lower than a preset threshold value, and stores the result in the RAM 113. Since the direction determination for a document image is a preprocessing for the actual character recognition, it is required to further increase the speed. On the other hand, although a certain number of characters are required for the direction determination, since it is only necessary to know the upright character direction, there is no problem in processing with a smaller character area than in the actual character recognition execution. Therefore, the threshold value that is the upper limit value of the cumulative total of the preset time factor values ​​may be smaller than the threshold value when performing the character recognition. In this embodiment, the threshold value is 50 as an example. In this case, the character area actually used for the direction determination is the character area in the range 1201 shown in FIG. 12 when expressed in a list.

[0065] In S1103, the system control unit 111 generates an execution image to be used for the direction determination process, which is composed only of the range of the character area set in S205, and stores the result in a storage unit such as the RAM 113 or the HDD 114. An example of the execution image for direction determination in this embodiment is shown in Fig. 13. An execution image 1301 shown in Fig. 13 is an execution image generated based on the character area of ​​the range 1201 shown in Fig. 12.

[0066] In S1104, the system control unit 111 determines the upright orientation of the document image based on the execution image for orientation determination generated in S1103, with reference to a storage unit such as the RAM 113 or the HDD 114. This process may be performed using a known technique, and may be performed using the process described in, for example, Japanese Patent No. 3727971. The system control unit 111 stores the angle of the upright character orientation obtained as the determination result in the RAM 113.

[0067] As a result, even in direction determination, which is a preprocessing step for actual character recognition, an image consisting only of character regions selected using the recognition priority and time factor values ​​is generated, and then direction determination is performed, thereby making it possible to further improve processing speed.

[0068] (Another embodiment of the present invention) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiment is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for implementing one or more of the functions.

[0069] It should be noted that the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]

[0070] 100: Reading device 101: Scanner unit 102: Communication unit 110: Image processing device 111: System control unit 112: ROM 113: RAM 114: HDD 115: Display unit 116: Input unit 117: Communication unit

Claims

1. A detection means for detecting a character region in an image; a priority setting means for classifying pixel blocks in each of the character regions based on size, and setting priorities for character recognition processing based on the number of pixel blocks belonging to each of the classified sizes; a time factor value setting means for setting, for each character region, a time factor value related to a processing time of a character recognition process based on information related to a size of a pixel block within the character region; and an execution order setting means for determining the character area and an execution order for character recognition processing based on the set priority and the set time factor value.

2. 2. The image processing apparatus according to claim 1, wherein said time factor value setting means sets said time factor value for each pixel block based on at least one of a height and a width of said pixel block.

3. 3. The image processing device according to claim 1, wherein the execution order setting means determines the character area for which character recognition processing is to be performed and the execution order based on the priority and the sum of the time factor values ​​set for each pixel block within the character area.

4. The image processing device according to any one of claims 1 to 3, characterized in that the time factor value setting means sets the time factor value based on information relating to the size of the pixel block obtained from the image without performing character recognition processing.

5. The time factor value setting means Calculating a virtual number of characters from a set of pixel blocks in the character region; Classifying the level of difficulty for each of the calculated virtual numbers of characters and setting a coefficient; 5. The image processing apparatus according to claim 1, wherein the virtual number of characters is set to 2 or more if the character is determined to be a touching character when the coefficient is set.

6. The execution order setting means The order of execution of the character recognition process is determined in descending order of the priority, and in the case where the priority is the same, in descending order of the time factor value; The image processing device according to any one of claims 1 to 5, characterized in that the character area for which character recognition processing is performed is a character area for which the sum of the time factor values ​​accumulated in order of execution does not exceed a predetermined threshold value.

7. The time factor value setting means An image processing device according to any one of claims 1 to 6, characterized in that when the time factor value of the set character region is greater than a predetermined threshold value, the pixel blocks that increase the time factor value are identified and deleted from the character region, and the time factor value of the character region is set.

8. An image processing device as described in any one of claims 1 to 6, characterized in that when the time factor value of the set character region is greater than a predetermined threshold value, the pixel block that increases the time factor value is identified and separated from the character region, and the priority and the time factor value are set for each separated character region.

9. The time factor value setting means 9. The image processing apparatus according to claim 1, wherein the time factor value is set differently depending on at least one of an algorithm for character recognition processing and a target language.

10. an image generating means for generating an execution image used for direction determination based on the set priority and the set time factor value; a direction determining means for determining a direction of the image using the generated execution image, 10. The image processing apparatus according to claim 1, further comprising: a step of rotating the image and executing character recognition processing based on a result of the determination by said direction determination means.

11. an execution area setting means for setting the character area for executing character recognition processing related to direction determination based on the set priority and the set time factor value; 11. The image processing apparatus according to claim 10, wherein the image generating means generates the execution image configured of the character area set by the execution area setting means.

12. A detection means for detecting a character region in an image; a priority setting means for classifying pixel blocks in each of the character regions based on size, and setting priorities for character recognition processing based on the number of pixel blocks belonging to each of the classified sizes; a time factor value setting means for setting, for each character region, a time factor value related to a processing time of a character recognition process based on information related to a size of a pixel block within the character region; an image generating means for generating an execution image used for direction determination based on the set priority and the set time factor value; and a direction determining unit for determining a direction of the image using the generated execution image.

13. an execution area setting means for setting the character area in which character recognition processing is to be performed based on the set priority and the set time factor value; 13. The image processing apparatus according to claim 12, wherein said image generating means generates said execution image constituted by said character area set by said execution area setting means.

14. An image processing method by an image processing device, comprising: a detection step of detecting a character region in the image; a priority setting step of classifying pixel blocks in each of the character regions based on size, and setting priorities for character recognition processing based on the number of pixel blocks belonging to each of the classified sizes; a time factor value setting step of setting, for each character region, a time factor value related to a processing time of a character recognition process based on information related to a size of a pixel block in the character region; and determining the character areas and an execution order for which character recognition processing is to be performed based on the set priority and the set time factor value.

15. An image processing method by an image processing device, comprising: a detection step of detecting a character region in the image; a priority setting step of classifying pixel blocks in each of the character regions based on size, and setting priorities for character recognition processing based on the number of pixel blocks belonging to each of the classified sizes; a time factor value setting step of setting, for each character region, a time factor value related to a processing time of a character recognition process based on information related to a size of a pixel block in the character region; an image generating step of generating an execution image to be used for direction determination based on the set priority and the set time factor value; and a direction determination step of determining a direction of the image using the generated execution image.

16. The computer of the image processing device A detection step of detecting a character region in the image; a priority setting step of classifying pixel blocks in each of the character regions based on size, and setting priorities for character recognition processing based on the number of pixel blocks belonging to each of the classified sizes; a time factor value setting step of setting, for each character region, a time factor value related to a processing time of a character recognition process based on information related to a size of a pixel cluster in the character region; and an execution order setting step of determining the character area and the execution order for which character recognition processing is to be performed based on the set priority and the set time factor value.

17. The computer of the image processing device A detection step of detecting a character region in the image; a priority setting step of classifying pixel blocks in each of the character regions based on size, and setting priorities for character recognition processing based on the number of pixel blocks belonging to each of the classified sizes; a time factor value setting step of setting, for each character region, a time factor value related to a processing time of a character recognition process based on information related to a size of a pixel cluster in the character region; an image generating step of generating an execution image used for direction determination based on the set priority and the set time factor value; and a direction determination step of determining a direction of the image using the generated execution image.

Images