Image processing device, image processing method, and program

The image processing device improves OCR accuracy by classifying document images based on character region characteristics and generating tailored learning data, addressing the accuracy issues in diverse degraded image scenarios.

JP2024163533AInactive Publication Date: 2024-11-22CANON KK
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023079242
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-12
Publication Date
2024-11-22
Estimated Expiration
Not applicable · inactive patent

Smart Images

  • Figure 2024163533000001_ABST
    Figure 2024163533000001_ABST
Patent Text Reader

Abstract

To improve OCR accuracy for a document image.SOLUTION: An image processing device according to the present invention is an image processing device for obtaining a learning model for character recognition, and includes: classifying means for classifying a document image on the basis of a characteristic of a character region in the document image obtained by reading a document; generation means for generating learning data being paired with correct answer data for each characteristic of the character region of the classified document image; and learning means for learning the learning model by using the learning data generated for each characteristic of the character region.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] Conventionally, there is a mechanism for performing character recognition (OCR: Optical Character Recognition) processing on document images obtained by scanning documents, using a learning model for character recognition that has been subjected to machine learning. Patent Document 1 discloses a technology for easily generating learning data for images (degraded images) in various degraded states, such as images with some characters blurred, as a technology for obtaining a learning model with higher accuracy. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-33721 Summary of the Invention [Problem to be solved by the invention]

[0004] Among the various degraded images, when a learning model is trained using different types of degraded images, such as a degraded image with a shadow in one part, a degraded image with whiteout, and a degraded image with a shadow in a part larger than the part, the OCR accuracy of the learning model is reduced. Therefore, when training the learning model, it is necessary to use appropriate types of degraded images together. [Means for solving the problem]

[0005] An image processing device according to one aspect of the present disclosure is an image processing device for obtaining a learning model for character recognition, and is characterized by having a classification means for classifying a document image obtained by reading the document based on features of character areas in the document image, a generation means for generating learning data that pairs with ground truth data for each feature of the character areas of the classified document image, and a learning means for learning the learning model using the learning data generated for each feature of the character areas. Effect of the Invention

[0006] According to the technology of the present disclosure, it is possible to improve the OCR accuracy for document images. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 illustrates an example of a configuration of an information processing system. [Diagram 2] 2 is a block diagram showing an example of a functional configuration of an information processing device; [Diagram 3] 13 is a flowchart showing the flow of a learning process for a character recognition model. [Figure 4] 13 is a flowchart showing a detailed flow of a first image analysis process. [Diagram 5] 1A and 1B are diagrams showing a target image of a first image analysis process and an example of a processing result. [Figure 6] FIG. 13 is a diagram showing an example of analysis for each class of pixel values ​​of pixels constituting a partial image. [Figure 7] FIG. 11 is a diagram showing an example of a determination condition in a first image classification. [Figure 8] 13 is a flowchart showing a detailed flow of a second image analysis process. [Figure 9] FIG. 11 is a diagram for explaining the second image analysis process. [Figure 10] FIG. 11 is a diagram showing an example of a determination condition in a second image classification. [Figure 11] 13 is a flowchart showing the flow of character recognition processing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, embodiments of the technology of the present disclosure will be described with reference to the drawings. Note that the following embodiments do not limit the technology of the present disclosure, and all configurations described in the following embodiments are not necessarily essential means for solving the problems of the present disclosure. The technology of the present disclosure is not limited to the following embodiments, and various modifications and changes are possible within the scope of the gist of the technology.

[0009] <Embodiment 1> [Hardware configuration] 1 is a diagram showing an example of the hardware configuration of an information processing system according to this embodiment. The information processing system according to this embodiment includes a copying machine 100 and an information processing device 110.

[0010] The copier 100 includes a scanner 101 and a copier-side communication unit 102. The scanner 101 scans a document and generates document image data (hereinafter also referred to as a document image). The copier-side communication unit 102 performs communication processing with external devices including an information processing device 110 via a network.

[0011] The information processing device 110 includes a system control unit 111, a ROM 112, a RAM 113, a HDD 114, a display unit 115, an input unit 116, and an information processing device side communication unit 117. The system control unit 111 is composed of an arithmetic unit such as a CPU (Central Processing Unit), and reads out a control program stored in a ROM (Read Only Memory) 112 to execute various processes. The RAM (Random Access Memory) 113 is used as a temporary storage area such as a main memory and a work area for the system control unit 111. The HDD (Hard Disk Drive) 114 is a large-capacity storage device that stores various data and various programs. The system control unit 111 reads out a program stored in the ROM 112 or the HDD 114, expands it in the RAM 113, and executes it to realize the functions and processes of the information processing device 110 described later. The information processing device side communication unit 117 executes communication processes with external devices including the copier 100 via a network. The display unit 115 displays various information. The input unit 116 has a keyboard and a mouse, and accepts various operations by the user. The display unit 115 and the input unit 116 may be provided integrally, such as a touch panel. The display unit 115 may project an image using a projector, and the input unit 116 may recognize the position of a fingertip relative to the projected image using a camera.

[0012] In this embodiment, a scanner 101 of a copier 100 scans a paper document such as a form and generates a document image. The document image is transmitted to an information processing device 110 by a copier-side communication unit 102. In the information processing device 110, an information processing device-side communication unit 117 receives the document image and stores the received document image in a storage device such as an HDD 114. The information processing device 110 can also be called an image processing device because it processes images such as document images.

[0013] 1 is an example of a configuration for realizing this embodiment, and for example, some functions of the display unit 115 and the input unit 116 may be included in the copier 100. Also, the copier 100 and the information processing device 110 may be integrated into one configuration.

[0014] [Software configuration] 2 is a diagram showing an example of the software configuration of the information processing device 110. The information processing device 110 includes an acquisition unit 201, a partial image generation unit 202, an image analysis unit 203, an image classification unit 204, a learning data generation unit 205, a model learning unit 206, a model selection unit 207, and a character recognition unit 208.

[0015] The acquiring unit 201 acquires the document image received from the copier 100 via the communication unit 102 of the copier 100 and the communication unit 117 of the information processing device 110. Note that the data of the document image received from the copier 100 may be stored in the HDD 114 of the information processing device 110, and the acquiring unit 201 may acquire the data of the document image stored in the HDD 144.

[0016] The partial image generating unit 202 performs a process of cutting out a part from the entire document image represented by the document image data, and generates an image of the part cut out from the entire document image as a partial image. The image analyzing unit 203 executes an image analysis process for analyzing the image state of the document image data. In the image analysis process, for example, the number of pixels of each pixel value of the partial image is calculated and the bias of the pixel number of each pixel value is analyzed, or the number of segments in the partial image and the interval between adjacent segments are estimated to calculate the degree of connection of character strings existing in the partial image. The image classifying unit 204 classifies the image based on the image analysis result output by the image analyzing unit 203.

[0017] The learning data generating unit 205 generates a set of image data for learning and correct answer data paired with the image data as learning data. The correct answer data is assumed to be data specified by a user operation. The model learning unit 206 trains a character recognition model for performing OCR, which is a learning model for character recognition, by performing machine learning using the learning data. The character recognition model performs character recognition of, for example, printed characters. The model selecting unit 207 selects a character recognition model (learning model) according to the characteristics of a character area in a document image to be processed from a plurality of character recognition models (learning models) learned by the model learning unit 206. The character recognition unit 208 executes OCR processing on the document image to be processed using the character recognition model selected by the model selecting unit 207.

[0018] [Learning process for character recognition model] Fig. 3 is a flowchart showing the flow of processing for learning a character recognition model. The processing of S301 to S306, S308, and S309 shown in the flowchart of Fig. 3 is realized by system control unit 111 reading a predetermined program from ROM 112 or the like, expanding it in RAM 113, and executing it. Note that HDD 114 stores data of document images used in the learning processing of the character recognition model. In the following explanation, the symbol "S" means a step.

[0019] In S301, the acquisition unit 201 acquires document image data for learning processing of a character recognition model from the HDD 114. At this time, property information such as the bit depth of the image is also acquired at the same time. Note that the document image data is, for example, data of a document image obtained by reading a document.

[0020] In S302, the partial image generating unit 202 generates a partial image from the document image represented by the data of the document image acquired in S301. The partial image is generated, for example, by projecting the document image, cutting out a series of character strings by performing line cutting to identify an area that is likely to be a line. In other words, the partial image can also be said to be an image in units of lines. The method of generating the partial image is not limited to the above line cutting. When the format of the document image is determined in advance, it is considered that lines exist in specific areas, so that the partial image may be generated by, for example, generating an image corresponding to a specific area in the document image specified by coordinate information as the partial image.

[0021] In S303, the image analysis unit 203 executes a first image analysis process on the partial image generated in S302. As a result, an analysis result of the partial image generated in S302 is obtained. Details of the first image analysis process will be described later with reference to the drawings.

[0022] In S304, the image classification unit 204 performs a first image classification based on the analysis result obtained in S303. This makes it possible to create an image data set classified according to the characteristics of the character regions in the document image. The details of the first image classification will be described later with reference to the drawings.

[0023] In S305, the image analysis unit 203 executes a second image analysis process on the partial image generated in S302. As a result, an analysis result of the partial image generated in S302 is obtained. Details of the second image analysis process will be described later with reference to the drawings.

[0024] In S306, the image classification unit 204 performs a second image classification based on the analysis result obtained in S305. This makes it possible to create an image data set classified according to the characteristics of the character regions in the document image. The details of the second image classification will be described later with reference to the drawings.

[0025] By repeating the processes in S301 to S306, image data classified according to the features of the character regions in the learning document images is created. A large number of image data are collected to create an image dataset.

[0026] In S307, correct answer data corresponding to the image data set is created. The correct answer data is a character string written in each image, and is character string information indicating the character string specified by a user operation. The character string information includes character code, character size, and coordinate information.

[0027] In S308, the learning data generation unit 205 generates learning data that is a pair of the created image data and the created supervised data. That is, the learning data generation unit 205 generates learning data that is a pair of supervised data for each feature of the character region of the classified partial image. The above learning data may be learning data for all combinations of conditions A to C and conditions D to F, or learning data for some combinations of conditions A to C and conditions D to F. The above learning data may be learning data for each of conditions A to C, or learning data for each of conditions D to F. A learning data set is created by collecting a large number of learning data.

[0028] In S309, the model learning unit 206 learns a character recognition model using the learning data set generated for each feature of the character region in S308. As a result, a character recognition model (learned model) trained for each feature of the character region is acquired.

[0029] [First image analysis process] FIG. 4 is a flowchart showing a detailed flow of the first image analysis process (S303). FIGS. 5(a) to (d) are diagrams for explaining the first image analysis process. FIGS. 6(a) and (b) are diagrams showing examples of distribution data showing bias in each class of pixel values ​​of pixels constituting a partial image. In FIGS. 5(b) and (d), pixel values ​​are shown horizontally, and the number of pixels at each pixel value is shown vertically. The number of pixels indicates the number of pixels showing the corresponding pixel value. FIG. 6(a) shows the case of the partial image shown in FIG. 5(b), and FIG. 6(b) shows the case of the partial image shown in FIG. 5(d).

[0030] In S401, the image analysis unit 203 determines whether the partial image generated in S302 is a grayscale image. This determination is made using the property information of the image acquired in S301. For example, if the property information indicates that the number of channels of the image is 1 and the bit depth is 8, the partial image to be analyzed is determined to be a grayscale image. If the determination result indicates that the partial image is not a grayscale image (NO in S401), the process proceeds to S402. If the determination result indicates that the partial image is a grayscale image (YES in S401), the process proceeds to S403.

[0031] In S402, the image analysis unit 203 generates a grayscale image by grayscaling the partial image generated in S302. In other words, the image analysis unit 203 generates a grayscale image by performing grayscale processing on the partial image acquired from the document image. For example, when the color of a partial image is expressed by a color value (RGB value) in an RGB color space, a conversion method is adopted for grayscaling, in which a weighting operation is performed on the RGB value, and the obtained value is set as a gray color value (gray value) corresponding to the RGB value of the color. An example of the weighting operation is NTSC conversion, which converts the RGB value by 0.299R+0.587G+0.114B.

[0032] In S403, the image analysis unit 203 calculates and acquires the number of pixels of each pixel value for the partial image (grayscale image) generated in S302. Alternatively, the image analysis unit 203 calculates and acquires the number of pixels of each pixel value for the partial image (grayscale image) generated in S402. That is, the image analysis unit 203 calculates and acquires the number of pixels having each pixel value for the partial image based on the pixel values ​​of the pixels constituting the partial image. FIG. 5(b) shows an example of the relationship between each pixel value and the number of pixels for the partial image (grayscale image) shown in FIG. 5(a). FIG. 5(d) shows an example of the relationship between each pixel value and the number of pixels for the partial image (grayscale image) shown in FIG. 5(c).

[0033] In S404, the image analysis unit 203 analyzes the bias of the number of pixels of each pixel value based on the number of pixels of each pixel value acquired in S403. Here, the frequency distribution is analyzed, and the frequency, relative frequency, cumulative frequency, and cumulative relative frequency are calculated with a predetermined pixel value range as a class. For example, the pixel value range is set to one class for each of 0 to 50, 51 to 190, 191 to 200, 201 to 225, 226 to 250, and 251 to 255. Based on the number of pixels of each pixel value shown in FIG. 5(b), the frequency, relative frequency, cumulative frequency, and cumulative relative frequency are calculated for each of the classes of 0 to 50, 51 to 190, 191 to 200, 201 to 225, 226 to 250, and 251 to 255, as shown in FIG. 6(a). Based on the number of pixels for each pixel value shown in Fig. 5(d), as shown in Fig. 6(b), the frequency, relative frequency, cumulative frequency, and cumulative relative frequency are calculated for each class of 0 to 50, 51 to 190, 191 to 200, 201 to 225, 226 to 250, and 251 to 255. That is, the image analysis unit 203 obtains distribution data indicating the bias of each pixel value of the pixels constituting the partial image as a feature of the character region in the document image.

[0034] [First image classification] FIG. 7 is a diagram showing an example of the judgment conditions in the first image classification (S304).

[0035] In the first image classification, first, the conditions for classifying partial images are determined in advance as conditions related to information obtained from distribution data indicating frequency distribution, and it is determined which condition the distribution data acquired in S303 applies to. The partial images are classified according to the determination result. For example, as shown in FIG. 7, partial images that satisfy the condition that the relative frequency is within a range of 0.6 to 1.0 in a class in which the pixel value is in the range of 251 to 255 are classified into an image dataset A. Partial images that satisfy the condition that the relative frequency is within a range of 0.5 to 1.0 in a class in which the pixel value is in the range of 51 to 200 are classified into an image dataset B. Partial images that satisfy the condition that the relative frequency is within a range of 0.5 to 1.0 in a class in which the pixel value is in the range of 191 to 255 are classified into an image dataset C. Note that partial images that do not satisfy the above conditions may be classified into an image dataset G different from the image datasets A to C and used for learning, or may not be used for learning as an image dataset outside the image dataset.

[0036] For example, the relationship between each pixel value and the number of pixels shown in Fig. 5(b) satisfies the conditions of image dataset A, so the partial image generated in S302 is classified into image dataset A. The relationship between each pixel value and the number of pixels shown in Fig. 5(d) satisfies the conditions of image dataset C, so the partial image generated in S302 is classified into image dataset C. Note that the conditions shown in Fig. 7 are only an example of conditions for classifying partial images, and do not necessarily have to be these conditions.

[0037] [Second image analysis process] Fig. 8 is a flowchart showing a detailed flow of the second image analysis process (S305). Note that the partial images shown in Fig. 5(c) and Fig. 5(d) are examples of line images.

[0038] In S801, the image analysis unit 203 regards the partial image generated in S302 as a line image, and acquires information about the line image. Here, the width and height of the partial image are acquired as information indicating the line width, which is the width of the line image, and the line height, which is the height of the line image.

[0039] In S802, the image analysis unit 203 estimates the number of characters that can exist in a line and the width of the characters based on the information acquired in S801. The width of one character is calculated from the line height to estimate the number of characters that can exist in a line. In this case, estimation is performed for both the case where all characters in the line are half-width alphanumeric characters and the case where all characters are full-width characters. When all characters in the line are full-width characters, as shown in FIG. 9(a), for example, all characters have the same width and height. When all characters in the line are half-width characters, as shown in FIG. 9(b), for example, all characters have the same width and height.

[0040] In S803, the image analysis unit 203 estimates the number of separators in a line and the spacing between adjacent separators based on the information on the number of characters and the width of the characters acquired in S801. The number of characters in half-width alphanumeric characters and full-width characters acquired in S801 is averaged, and this value is used as an estimate of the number of separators. In addition, the width of characters in half-width alphanumeric characters and full-width characters acquired in S801 is averaged, and this value is used as an estimate of the spacing between adjacent separators. In other words, the image analysis unit 203 acquires estimated information that estimates the number of separators and the spacing.

[0041] In S804, the image analysis unit 203 acquires the number of partitions in the partial image (row image) and the spacing between adjacent partitions. A horizontal projection of the row image is taken to acquire blank positions, and the number of partitions and the spacing between adjacent partitions are acquired. Figures 9(c) and 9(d) are diagrams showing examples of spacing between adjacent partitions acquired from the row image when the row image is projected horizontally. From the partial image (row image) shown in Figure 9(c), 14 is acquired as the number of partitions. From the partial image (row image) shown in Figure 9(d), 9 is acquired as the number of partitions.

[0042] In S805, the image analysis unit 203 compares the number of segments estimated in S803 with the number of segments acquired in S804. The image analysis unit 203 also compares the interval between adjacent segments estimated in S803 with the interval between adjacent segments acquired in S804. That is, the image analysis unit 203 acquires a comparison result regarding the number of segments and a comparison result regarding the interval between adjacent segments.

[0043] [Second Image Classification] FIG. 10 is a diagram showing an example of the judgment conditions in the second image classification (S306).

[0044] In the second image classification, the target partial image is classified based on the comparison result acquired in S805. For example, a classification table for classifying partial images is used to determine which condition the information on the degree of connection obtained from the number of segments and the intervals applies to. The partial image is classified according to the determination result. For example, as shown in FIG. 10, a partial image that satisfies the condition that the degree of connection is small is classified into image data set D. That is, a partial image that satisfies the condition that the difference in intervals is within a threshold (first threshold) range and the ratio of the number of intervals is less than a threshold (second threshold) is classified into image data set D. A partial image that satisfies the condition that the degree of connection is medium is classified into image data set E. That is, a partial image that satisfies the condition that the difference in intervals is within a threshold (first threshold) range and the ratio of the number of intervals is equal to or greater than a threshold (second threshold) is classified into image data set E. A partial image that satisfies the condition that the degree of connection is large is classified into image data set F. That is, a partial image that satisfies the condition that the difference in intervals is equal to or greater than a threshold (first threshold) and the ratio of the number of intervals is equal to or greater than a threshold (second threshold) is classified into image data set F.

[0045] Here, we will explain the case where the estimated number of segments obtained in S803 is 15, the estimated distance between adjacent segments (spacing between segments) obtained in S803 is 30, and the following values ​​are obtained in S804.

[0046] [Table 1] The conditions for determining that an image dataset is E are that the most common difference in partition intervals (difference in intervals) is 15 or less, and the proportion of the number of intervals is 80% or more.

[0047] In the above table, the total number of intervals is 14 (=12+1+1), the most common division width is 20, and there are 12 of them.

[0048] In this case, the difference in intervals is 15 or less (|30-20|=10), and the number exists in more than 80% (10 / 14=0.85).

[0049] Therefore, since the condition for determining that it is image data set E is met, it is determined to be image data set E.

[0050] As shown in FIG. 10, conditions regarding the number of segments and spacing are determined in advance as the degree of coupling, and the segments are classified according to the degree to which they apply.

[0051] By classifying the line images, which are partial images, according to the degree of connection in this manner, it is possible to create an image data set according to the degree of connection of characters present in the line.

[0052] [Character recognition processing] Fig. 11 is a flowchart showing the flow of character recognition processing. Note that the processing of S1101 to S1105 shown in the flowchart of Fig. 11 is realized by the system control unit 111 reading a predetermined program from the ROM 112 or the like, expanding it in the RAM 113, and executing it. In the character recognition processing shown in Fig. 11, the character recognition model learned in the learning processing shown in Fig. 3 is used.

[0053] In S1101, the acquisition unit 201 acquires from the HDD 114 data of a document image to be subjected to character recognition processing.

[0054] In S1102, the partial image generating unit 202 generates a partial image from the document image represented by the document image data acquired in S1101. The partial image is generated, for example, as in S302 above, by performing line cutting to identify an area that is likely to be a line by projecting the document image, and cutting out a series of character strings. The method of generating the partial image is not limited to the above line cutting. When the format of the document image is determined in advance, it is considered that a line exists in a specific area, so that, for example, a method of generating an image corresponding to a specific area in the document image specified by coordinate information as a partial image may be used as the method of generating the partial image.

[0055] In S1103, the image analysis unit 203 performs image analysis on the partial image generated in S1102. The image analysis in S1103 is the same processing as the first image analysis in S303 and the second image analysis in S305. Alternatively, the image analysis in S1103 is the same processing as the first image analysis in S303. The image analysis unit 203 acquires an analysis result of the partial image generated in S1102 by performing the image analysis processing.

[0056] In S1104, the model selection unit 207 selects a character recognition model corresponding to the analysis result acquired in S1103. The selection of the character recognition model is a process using the same judgment conditions as the first image classification in S304. Alternatively, the selection of the character recognition model is a process using the same judgment conditions as the second image classification in S306. The classification of the partial images into each image data set under predetermined conditions in S304 or S306 is replaced with a process of selecting a character recognition model trained with each image data set. As a result, from among the multiple character recognition models, a character recognition model optimal for each feature of the character region in the document image is selected. Note that, in the case where an image analysis result that does not correspond to any of the image data sets A to F is obtained in S1103, a character recognition model trained using an image data set that is not classified according to the feature of the character region in the document image is selected as in the conventional case.

[0057] In S1105, the character recognition unit 208 executes character recognition processing on the document image acquired in S1001, using the character recognition model selected in S1104.

[0058] As described above, in this embodiment, a partial image is acquired from a document image, an image data set is created for each feature of a character region in the partial image obtained by image analysis of the partial image, and a character recognition model is learned using the image data set for each feature of the character region. For the document image to be OCR processed, OCR processing is performed using a character recognition model corresponding to each feature of the character region. This makes it possible to acquire an appropriate character string as the character recognition result. In other words, even if the document image is a degraded image, characters can be recognized by performing OCR processing using a character recognition model corresponding to each feature of the character region in the document image. This makes it possible to improve the OCR accuracy for the document image.

[0059] <Other embodiments> In the above, a case has been described in which a character recognition model is trained for each feature of a character region using a training data set generated for each feature of the character region by performing a first image analysis process and image classification, and a second image analysis process and image classification. However, the present invention is not limited to this. A first image analysis process and a first image classification may be performed to create a training data set corresponding to an image data set classified for each feature of the character region in a document image, and a character recognition model may be trained using the training data set generated for each feature of the character region.

[0060] Although the above describes a case where a character recognition model is selected using the first and second image analyses and the conditions of the first and second image classifications, and character recognition processing is performed using the selected character recognition model, the present invention is not limited to this. It is also possible to perform a first image analysis, a character recognition model is selected using the conditions of the first image classification, and character recognition processing is performed using the selected character recognition model.

[0061] The present disclosure can also be realized by a process in which a program for implementing one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that implements one or more functions.

[0062] The disclosure of this embodiment includes the following configuration examples.

[0063] (Configuration 1) An image processing device for obtaining a learning model for character recognition, comprising: A classification means for classifying a document image obtained by reading the document based on characteristics of a character area in the document image; a generating means for generating learning data paired with a correct answer data for each feature of the character region of the classified document image; a learning means for learning the learning model by using the learning data generated for each feature of the character region; 13. An image processing device comprising:

[0064] (Configuration 2) The document image may further include an analysis unit for analyzing the document image to obtain characteristics of the character region. The classification means classifies the document image based on the acquired characteristics of the character region. 2. The image processing device according to claim 1,

[0065] (Configuration 3) The document image further includes an acquisition unit for acquiring a partial image in units of lines from the document image, the analyzing means acquires distribution data indicating a bias of each pixel value of pixels constituting the acquired partial image as a feature of the character region; The classification means classifies the partial images according to characteristics of the character regions based on the acquired distribution data, The learning means learns the learning model by using the learning data generated for each feature of the character region of the classified partial image. 3. The image processing device according to configuration 2.

[0066] (Configuration 4) The document image further includes an acquisition unit for acquiring a partial image in units of lines from the document image, the analyzing means acquires, as a feature of the character region, character string information indicating a number of segments into which the acquired partial image is divided by characters and a distance between adjacent segments in the partial image; The classification means classifies the partial images according to characteristics of the character regions based on the acquired character string information, The learning means learns the learning model by using the learning data generated for each feature of the character region of the classified partial image. 3. The image processing device according to configuration 2.

[0067] (Configuration 5) The document image further includes an acquisition unit for acquiring a partial image in units of lines from the document image, the analysis means acquires, as features of the character region, distribution data indicating a bias for each pixel value of pixels constituting the acquired partial image, and character string information indicating a number of segments into which the acquired partial image is divided by characters and an interval between adjacent segments in the partial image; the classification means classifies the partial images according to features of the character regions based on the acquired distribution data and the acquired character string information; The learning means learns the learning model by using the learning data generated for each feature of the character region of the classified partial image. 3. The image processing device according to configuration 2.

[0068] (Configuration 6) The analysis means obtaining estimated information that estimates the number of divisions and the intervals based on the number of characters that may be present in the partial image and the horizontal sizes of the characters that may be present in the partial image; comparing the number of segments of the character string information with the number of segments of the estimated information, and comparing the interval of the character string information with the interval of the estimated information; The classification means classifies the partial images according to characteristics of the character regions based on a comparison result obtained by the comparison by the analysis means. 6. The image processing device according to configuration 4 or 5.

[0069] (Configuration 7) The analysis means If the document image is not a grayscale image, grayscale processing the document image to generate a grayscale image; The generated grayscale image is analyzed to obtain characteristics of the character region. 7. The image processing device according to any one of configurations 2 to 6,

[0070] (Configuration 8) The learning model recognizes printed characters. 8. The image processing device according to any one of configurations 1 to 7.

[0071] (Configuration 9) An image processing device that performs character recognition using a learning model for character recognition, a selection means for selecting a learning model according to the characteristics of the character region in the document image to be processed from a plurality of learning models trained for each characteristic of the character region in the learning document image; a character recognition means for performing character recognition on the document image to be processed by using the selected learning model; 13. An image processing device comprising:

[0072] (Configuration 10) The document image may further include an analysis unit for analyzing the document image to obtain characteristics of the character region. The selection means selects the learning model based on the acquired characteristics of the character region. 10. The image processing device according to configuration 9.

[0073] (Configuration 11) The image processing method further includes: acquiring a partial image in units of lines from the document image to be processed; the analyzing means acquires distribution data indicating a bias of pixel values ​​of pixels constituting the partial image as a feature of the character region; the selection means selects the learning model corresponding to the partial image classified by the feature of the character region based on the acquired distribution data; The character recognition means performs character recognition on the document image to be processed by using the learning model selected for each feature of the character region of the classified partial image. 11. The image processing device according to configuration 10.

[0074] (Configuration 12) The document image further includes an acquisition unit for acquiring a partial image in units of lines from the document image, the analyzing means acquires, as a feature of the character region, character string information indicating a number of segments into which the acquired partial image is divided by characters and a distance between adjacent segments in the partial image; The selection means selects the learning model corresponding to the partial images classified by features of the partial images based on the acquired character string information, The character recognition means performs character recognition on the document image to be processed by using the learning model selected for each feature of the character region of the classified partial image. 11. The image processing device according to configuration 10.

[0075] (Configuration 13) The document image further includes an acquisition unit for acquiring a partial image in units of lines from the document image, the analysis means acquires, as features of the character region, distribution data indicating a bias for each pixel value of pixels constituting the acquired partial image, and character string information indicating a number of segments into which the acquired partial image is divided by characters and an interval between adjacent segments in the partial image; The character recognition means performs character recognition on the document image to be processed by using the learning model selected for each feature of the character region of the classified partial image. 11. The image processing device according to configuration 10.

[0076] (Configuration 14) The analysis means obtaining estimated information that estimates the number of divisions and the intervals based on the number of characters that may be present in the partial image and the horizontal sizes of the characters that may be present in the partial image; comparing the number of segments of the character string information with the number of segments of the estimated information, and comparing the interval of the character string information with the interval of the estimated information; The selection means selects the learning model for each feature of the character region based on a comparison result obtained by the comparison by the analysis means. 14. The image processing device according to claim 12 or 13.

[0077] (Configuration 15) The analysis means If the document image is not a grayscale image, grayscale processing the document image to generate a grayscale image; The generated grayscale image is analyzed to obtain characteristics of the character region. 15. The image processing device according to any one of configurations 10 to 14.

[0078] (Configuration 16) The learning model recognizes printed characters. 16. The image processing device according to any one of configurations 9 to 15.

[0079] (Configuration 17) 1. An image processing method for obtaining a learning model for character recognition, comprising the steps of: A classification step of classifying the document image based on characteristics of a character region in the document image obtained by reading the document; a generating step of generating learning data paired with ground truth data for each feature of the character region of the classified document image; a learning step of learning the learning model by using the learning data generated for each feature of the character region; 13. An image processing method comprising:

[0080] (Configuration 18) An image processing method for performing character recognition using a learning model for character recognition, comprising: a selection step of selecting a learning model according to the features of the character region in the document image to be processed from a plurality of learning models trained for each feature of the character region in the learning document image; a character recognition step of performing character recognition on the document image to be processed by using the selected learning model; 13. An image processing method comprising:

[0081] (Configuration 19) 18. A program for causing a computer to execute the image processing method according to claim 17.

[0082] (Configuration 20) 20. A program for causing a computer to execute the image processing method according to claim 18. [Explanation of symbols]

[0083] 204 Image Classification Unit 205 Learning Data Generation Unit 206 Model Learning Department

Claims

1. An image processing device for obtaining a learning model for character recognition, comprising: A classification means for classifying a document image obtained by reading the document based on characteristics of a character area in the document image; a generating means for generating learning data paired with a correct answer data for each feature of the character region of the classified document image; a learning means for learning the learning model by using the learning data generated for each feature of the character region; 13. An image processing device comprising:

2. The document image may further include an analysis unit for analyzing the document image to obtain characteristics of the character region. The classification means classifies the document image based on the acquired characteristics of the character region.

2. The image processing device according to claim 1,

3. The document image further includes an acquisition unit for acquiring a partial image in units of lines from the document image, the analyzing means acquires distribution data indicating a bias of each pixel value of pixels constituting the acquired partial image as a feature of the character region; The classification means classifies the partial images according to characteristics of the character regions based on the acquired distribution data, The learning means learns the learning model by using the learning data generated for each feature of the character region of the classified partial image.

3. The image processing device according to claim 2.

4. The document image further includes an acquisition unit for acquiring a partial image in units of lines from the document image, the analyzing means acquires, as a feature of the character region, character string information indicating a number of segments into which the acquired partial image is divided by characters and a distance between adjacent segments in the partial image; The classification means classifies the partial images according to characteristics of the character regions based on the acquired character string information, The learning means learns the learning model by using the learning data generated for each feature of the character region of the classified partial image.

3. The image processing device according to claim 2.

5. The document image further includes an acquisition unit for acquiring a partial image in units of lines from the document image, the analysis means acquires, as features of the character region, distribution data indicating a bias for each pixel value of pixels constituting the acquired partial image, and character string information indicating a number of segments into which the acquired partial image is divided by characters and an interval between adjacent segments in the partial image; the classification means classifies the partial images according to features of the character regions based on the acquired distribution data and the acquired character string information; The learning means learns the learning model by using the learning data generated for each feature of the character region of the classified partial image.

3. The image processing device according to claim 2.

6. The analysis means obtaining estimated information that estimates the number of divisions and the intervals based on the number of characters that may be present in the partial image and the horizontal sizes of the characters that may be present in the partial image; comparing the number of segments of the character string information with the number of segments of the estimated information, and comparing the interval of the character string information with the interval of the estimated information; The classification means classifies the partial images according to characteristics of the character regions based on a comparison result obtained by the comparison by the analysis means.

5. The image processing device according to claim 4.

7. The analysis means If the document image is not a grayscale image, grayscale processing the document image to generate a grayscale image; The generated grayscale image is analyzed to obtain characteristics of the character region.

3. The image processing device according to claim 2.

8. The learning model recognizes printed characters.

2. The image processing device according to claim 1,

9. An image processing device that performs character recognition using a learning model for character recognition, a selection means for selecting a learning model according to the characteristics of the character region in the document image to be processed from a plurality of learning models trained for each characteristic of the character region in the learning document image; a character recognition means for performing character recognition on the document image to be processed by using the selected learning model; 13. An image processing device comprising:

10. The document image may further include an analysis unit for analyzing the document image to obtain characteristics of the character region. The selection means selects the learning model based on the acquired characteristics of the character region.

10. The image processing device according to claim 9,

11. The image processing method further includes: acquiring a partial image in units of lines from the document image to be processed; the analyzing means acquires distribution data indicating a bias of pixel values ​​of pixels constituting the partial image as a feature of the character region; the selection means selects the learning model corresponding to the partial image classified by the feature of the character region based on the acquired distribution data; The character recognition means performs character recognition on the document image to be processed by using the learning model selected for each feature of the character region of the classified partial image. The image processing device according to claim 10 .

12. The document image further includes an acquisition unit for acquiring a partial image in units of lines from the document image, the analyzing means acquires, as a feature of the character region, character string information indicating a number of segments into which the acquired partial image is divided by characters and a distance between adjacent segments in the partial image; The selection means selects the learning model corresponding to the partial images classified by features of the partial images based on the acquired character string information, The character recognition means performs character recognition on the document image to be processed by using the learning model selected for each feature of the character region of the classified partial image. The image processing device according to claim 10 .

13. The document image further includes an acquisition unit for acquiring a partial image in units of lines from the document image, the analysis means acquires, as features of the character region, distribution data indicating a bias for each pixel value of pixels constituting the acquired partial image, and character string information indicating a number of segments into which the acquired partial image is divided by characters and an interval between adjacent segments in the partial image; The character recognition means performs character recognition on the document image to be processed by using the learning model selected for each feature of the character region of the classified partial image. The image processing device according to claim 10 .

14. The analysis means obtaining estimated information that estimates the number of divisions and the intervals based on the number of characters that may be present in the partial image and the horizontal sizes of the characters that may be present in the partial image; comparing the number of segments of the character string information with the number of segments of the estimated information, and comparing the interval of the character string information with the interval of the estimated information; The selection means selects the learning model for each feature of the character region based on a comparison result obtained by the comparison by the analysis means. The image processing device according to claim 12 .

15. The analysis means If the document image is not a grayscale image, grayscale processing the document image to generate a grayscale image; The generated grayscale image is analyzed to obtain characteristics of the character region. The image processing device according to claim 10 .

16. The learning model recognizes printed characters.

10. The image processing device according to claim 9,

17. 1. An image processing method for obtaining a learning model for character recognition, comprising the steps of: A classification step of classifying a document image obtained by reading the document based on characteristics of a character region in the document image; a generating step of generating learning data paired with ground truth data for each feature of the character region of the classified document image; a learning process of learning the learning model using the learning data generated for each feature of the character region; 13. An image processing method comprising:

18. An image processing method for performing character recognition using a learning model for character recognition, comprising: a selection step of selecting a learning model according to the features of the character region in the document image to be processed from a plurality of learning models trained for each feature of the character region in the learning document image; a character recognition step of performing character recognition on the document image to be processed by using the selected learning model; 13. An image processing method comprising:

19. A program for causing a computer to execute the image processing method according to claim 17.

20. A program for causing a computer to execute the image processing method according to claim 18.

Citation Information

Patent Citations

  • Information processing system, control method thereof, and program

    JP2021033721A