Image processing apparatus, image processing system, output apparatus, image processing method, and image processing program

Abstract

To provide an image processor, an image processing system, an output device, an image processing method, and an image processing program for improving the character recognition accuracy of a character string including a small-sized handwritten character.SOLUTION: The image processing apparatus 1 includes an acquisition processor 111 that acquires character image data, a character string extraction processor 112 that extracts a character string rectangle corresponding to a character string composed of a plurality of characters from the character image data, a one character extraction processor 113 that extracts a plurality of one character rectangles respectively corresponding to the plurality of characters from the character image data, a correction processor 114 that corrects a specific one character rectangle when the plurality of one character rectangles include the specific one character rectangle in which at least one of a position and a size satisfies a predetermined condition, and a recognition processor 115 that executes character recognition processing of the character string using the specific one character rectangle corrected by the correction processor 114.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a technique for performing image processing such as character recognition on an input image. [Background technology]

[0002] Conventionally, there are known techniques for recognizing (OCR processing) handwritten characters in documents, forms, etc. For example, there is known a technique for estimating handwritten portions and background portions from a scanned image, extracting contours existing within a processing target area of ​​the scanned image, and correcting the estimation result based on the coordinate position of the extracted contour, the coordinate position of the estimated handwritten portion, and the coordinate position of the estimated background portion (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-26170 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional technologies, when a string of handwritten characters contains small characters such as commas, periods, and punctuation marks, the small characters can be mistakenly recognized as larger characters. For example, when OCR is performed on the character string "￥99,600-" contained in an input image, the comma "," is mistakenly recognized as a "9," and "￥999600-" is output as the OCR result.

[0005] An object of the present disclosure is to provide an image processing device, an image processing system, an output device, an image processing method, and an image processing program that are capable of improving the character recognition accuracy of a string of characters that includes small handwritten characters. [Means for solving the problem]

[0006] An image processing device according to one aspect of the present disclosure includes an acquisition processing unit, a first extraction processing unit, a second extraction processing unit, a correction processing unit, and a recognition processing unit. The acquisition processing unit acquires character image data. The first extraction processing unit extracts character string rectangles corresponding to a character string consisting of a plurality of characters from the character image data. The second extraction processing unit extracts a plurality of single-character rectangles corresponding to each of the plurality of characters from the character image data. The correction processing unit corrects a specific single-character rectangle when the plurality of single-character rectangles include the specific single-character rectangle whose position and / or size satisfy a predetermined condition. The recognition processing unit performs character recognition processing on the character string using the specific single-character rectangle corrected by the correction processing unit.

[0007] An image processing system according to one aspect of the present disclosure includes the image processing device and a learning device that generates a trained model by performing machine learning using the learning data generated by the image processing device.

[0008] An output device according to another aspect of the present disclosure performs character recognition processing on an input image using the trained model generated by a training device, and outputs the character recognition result.

[0009] An image processing method according to another aspect of the present disclosure is an image processing method performed by one or more processors, which includes acquiring character image data, extracting character string rectangles from the character image data corresponding to a character string consisting of a plurality of characters, extracting a plurality of single-character rectangles from the character image data corresponding to each of the plurality of characters, correcting a specific single-character rectangle if the multiple single-character rectangles include one whose position and / or size satisfy predetermined conditions, and performing character recognition processing on the character string using the corrected specific single-character rectangle.

[0010] An image processing program according to another aspect of the present disclosure is an image processing program for causing one or more processors to execute the following steps: acquiring character image data; extracting character string rectangles from the character image data corresponding to a character string consisting of a plurality of characters; extracting a plurality of single-character rectangles from the character image data corresponding to each of the plurality of characters; if the plurality of single-character rectangles include a specific single-character rectangle whose position and / or size satisfy predetermined conditions, correcting the specific single-character rectangle; and performing character recognition processing of the character string using the corrected specific single-character rectangle. [Effects of the Invention]

[0011] According to the present disclosure, it is possible to provide an image processing device, an image processing system, an output device, an image processing method, and an image processing program that are capable of improving the character recognition accuracy of a string of characters that includes small-sized handwritten characters. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a functional block diagram showing the configuration of an image processing system according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is a diagram illustrating an example of an input image (receipt) according to an embodiment of the present disclosure. [Figure 3] FIG. 3 is a diagram illustrating an example of an input image (receipt) according to an embodiment of the present disclosure. [Figure 4] FIG. 4 is a diagram illustrating a specific example of character recognition processing according to an embodiment of the present disclosure. [Figure 5] FIG. 5 is a diagram illustrating a specific example of character recognition processing according to an embodiment of the present disclosure. [Figure 6] FIG. 6 is a diagram illustrating a specific example of character recognition processing according to an embodiment of the present disclosure. [Figure 7] FIG. 7 is a diagram illustrating a specific example of character recognition processing according to an embodiment of the present disclosure. [Figure 8]FIG. 8 is a flowchart showing an example of the procedure of character recognition processing executed in the image processing device according to an embodiment of the present disclosure. [Figure 9] FIG. 9 is a diagram illustrating a specific example of character recognition processing according to an embodiment of the present disclosure. [Figure 10] FIG. 10 is a diagram illustrating a specific example of character recognition processing according to an embodiment of the present disclosure. [Figure 11] FIG. 11 is a diagram illustrating a specific example of character recognition processing according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that the following embodiments are examples that embody the present disclosure and do not limit the technical scope of the present disclosure.

[0014] 1 is a block diagram showing a configuration of an image processing system 10 according to an embodiment of the present disclosure. The image processing system 10 includes an image processing device 1 and a learning device 2. The image processing device 1 is an information processing device that recognizes character strings included in an input image (image data), performs character recognition processing (OCR processing) on ​​the recognized character strings, and outputs the character recognition results. The learning device 2 is an information processing device that performs machine learning using input data (learning data) input from the image processing device 1, and generates a trained model for performing character recognition on the input image.

[0015] 1, the image processing device 1 includes a control unit 11, a storage unit 12, an operation display unit 13, and a communication unit 14. The image processing device 1 may be one or more cloud servers, or one or more physical servers.

[0016] The communication unit 14 is a communication interface that connects the image processing device 1 to a network N1 by wire or wirelessly and executes data communication with an external device (e.g., the learning device 2) via the network N1 in accordance with a predetermined communication protocol. The network N1 is configured, for example, by the Internet, a LAN, or the like.

[0017] The operation display unit 13 is a user interface that includes a display unit such as a liquid crystal display or an organic EL display that displays various information, and an operation unit such as a mouse, keyboard, or touch panel that accepts operations.

[0018] The storage unit 12 is a non-volatile storage unit such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory that stores various types of information. The storage unit 12 stores a control program such as a character recognition program (an example of an image processing program of the present disclosure) that causes the control unit 11 to execute a character recognition process described below. For example, the character recognition program is non-temporarily recorded on a computer-readable recording medium such as a CD or a DVD, and is read by a reading device (not shown) such as a CD drive or a DVD drive provided in the image processing device 1 and stored in the storage unit 12. The character recognition program may be distributed from a cloud server and stored in the storage unit 12.

[0019] The storage unit 12 also stores image data (scanned data, etc.) of documents and the like acquired from external devices.

[0020] FIG. 2 shows a receipt as an example of a document. As shown in FIG. 2, a receipt includes multiple items such as the issue date, destination, contact information of the issuer, and amount. For example, a user scans the receipt using a scanner, multifunction peripheral, or the like, and uploads the image data (input image) to the image processing device 1. Alternatively, a user may photograph the receipt using a camera on an operation terminal (e.g., a smartphone) and upload the image data to the image processing device 1. Upon acquiring the image data of the receipt, the control unit 11 stores the image data in the storage unit 12. In another embodiment, the control unit 11 may acquire a document file of a receipt created in an external device and store the document file in the storage unit 12.

[0021] The control unit 11 has control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various types of arithmetic processing. The ROM stores in advance control programs such as a BIOS and an OS that cause the CPU to execute various types of processing. The RAM stores various types of information and is used as a temporary storage memory (work area) for the various types of processing executed by the CPU. The control unit 11 controls the image processing device 1 by having the CPU execute various control programs that are pre-stored in the ROM or the storage unit 12.

[0022] However, conventional technologies have a problem in that, for example, when a string of handwritten characters includes small handwritten characters such as commas, periods, and punctuation marks, the small handwritten characters are mistakenly recognized as large characters. For example, as shown in FIG. 2, when OCR processing is performed on the string "￥99,600-" included in an input image, the comma "," may be mistakenly recognized as a "9," resulting in the output of "￥999600-" as the OCR result. Similarly, as shown in FIG. 3, when OCR processing is performed on the string "￥98,345-," the comma "," may be mistakenly recognized as a "1," resulting in the output of "￥981345-." In response to this problem, the image processing device 1 according to the present disclosure has a configuration capable of improving the character recognition accuracy of strings including small handwritten characters, as described below.

[0023] Specifically, as shown in Fig. 1, the control unit 11 includes various processing units such as an acquisition processing unit 111, a character string extraction processing unit 112, a single character extraction processing unit 113, a correction processing unit 114, a recognition processing unit 115, and an output processing unit 116. The control unit 11 functions as the various processing units by executing various processes in accordance with the character recognition program. Some or all of the processing units included in the control unit 11 may be configured with electronic circuits. The character recognition program may be a program for causing multiple processors to function as the various processing units.

[0024] The acquisition processing unit 111 acquires an input image. Specifically, the acquisition processing unit 111 acquires an image (character image data) to be subjected to character recognition. For example, the acquisition processing unit 111 acquires character image data of a form image including handwritten characters, such as a receipt shown in FIG. 2.

[0025] The character string extraction processing unit 112 extracts a character string rectangle corresponding to a character string consisting of multiple characters from the input image. Specifically, the character string extraction processing unit 112 executes a character string recognition process in which document part recognition processing is executed on the input image acquired by the acquisition processing unit 111 to recognize a character string consisting of multiple characters. The character string extraction processing unit 112 also sets a character string rectangle corresponding to the recognized character string. In other words, the character string extraction processing unit 112 recognizes a group of multiple characters as a character string rectangle. For example, in the input image shown in FIG. 2, the character string extraction processing unit 112 extracts a character string rectangle K1 (see FIG. 4) corresponding to a handwritten character string ("￥99,600-"). The character string extraction processing unit 112 is an example of a first extraction processing unit of the present disclosure.

[0026] The single character extraction processing unit 113 extracts a plurality of single character rectangles corresponding to each of a plurality of characters from the input image. Specifically, the single character extraction processing unit 113 executes a single character recognition process to recognize characters on a character-by-character basis in the input image acquired by the acquisition processing unit 111. The single character extraction processing unit 113 also extracts a single character rectangle corresponding to the recognized character. In other words, the single character extraction processing unit 113 recognizes the smallest unit of a character as a character string rectangle. Note that the single character extraction processing unit 113 may extract a plurality of single character rectangles corresponding to each of a plurality of characters from the character string extracted by the character string extraction processing unit 112.

[0027] For example, as shown in Fig. 4, the single-character extraction processing unit 113 extracts a plurality of single-character rectangles K2 corresponding to each character. Fig. 4 shows an example in which eight single-character rectangles K21 to K28 are extracted from a character string rectangle K1. The single-character extraction processing unit 113 is an example of a second extraction processing unit of the present disclosure.

[0028] If a specific character rectangle (hereinafter referred to as a specific character rectangle) whose position and size satisfy predetermined conditions is included in the multiple single-character rectangles, the correction processing unit 114 corrects the specific character rectangle. The correction processing unit 114 identifies a single-character rectangle of a small character that may be erroneously recognized as a specific character rectangle, and performs correction processing on the specific character rectangle.

[0029] Specifically, the correction processing unit 114 determines whether the multiple single-character rectangles include a single-character rectangle whose area is less than a predetermined area (first condition), and whether the multiple single-character rectangles include a single-character rectangle that is positioned at a position that is more than a predetermined distance away from the outer edge of the character string rectangle K1 (second condition).

[0030] Then, if the multiple single-character rectangles include a single-character rectangle whose area is less than a predetermined area (if the first condition is met) and which is located at a position more than a predetermined distance away from the outer edge of the character string rectangle K1 (if the second condition is met), the correction processing unit 114 identifies the single-character rectangle as a specific character rectangle and corrects the specific character rectangle.

[0031] In the example shown in FIG. 5, the correction processing unit 114 calculates the areas M1 to M8 of the eight single-character rectangles K21 to K28, respectively, and the average area Ma of the single-character rectangles K21 to K28. The correction processing unit 114 also calculates a reference area Mb (=Ma×F1) by multiplying the average area Ma by a correction coefficient F1 (e.g., F1=0.3). The correction processing unit 114 then identifies a single-character rectangle from among the areas M1 to M8 that is less than the reference area Mb. Here, the correction processing unit 114 identifies a single-character rectangle K24 (",") from among the areas M1 to M8 that is less than the reference area Mb. In this way, the correction processing unit 114 determines whether the single-character rectangles K21 to K28 include a single-character rectangle whose area is less than 30% of the average area. The correction coefficient F1 is not limited to 0.3 and is set to a value in the range of 0.1 to 0.3, for example.

[0032] 5, the correction processing unit 114 calculates the heights (distances) h1 to h8 of each of the eight single-character rectangles K21 to K28 from the top side of the character string rectangle K1, and the height H1 of the character string rectangle K1. The correction processing unit 114 also calculates a reference height H2 (=H1×F2) by multiplying the height H1 of the character string rectangle K1 by a correction coefficient F2 (e.g., F2=0.5). The correction processing unit 114 then identifies a single-character rectangle of the heights h1 to h8 that is equal to or greater than the reference height H2. Here, the correction processing unit 114 identifies a single-character rectangle K24 (",") of the height h4 that is equal to or greater than the reference height H2 (=H1×0.5) from the heights h1 to h8. In this way, the correction processing unit 114 determines whether the single-character rectangles K21 to K28 include a single-character rectangle whose height (distance) from the top side of the character string rectangle K1 is 50% or greater than the height of the character string rectangle K1. That is, the correction processing unit 114 identifies a single-character rectangle located in the lower half of the character string rectangle K1. The correction coefficient F2 is not limited to 0.5, and is set to a value in the range of 0.5 to 0.9, for example.

[0033] As described above, the correction processor 114 identifies specific character rectangles that satisfy the first and second conditions from among the multiple single-character rectangles K2 included in the character string rectangle K1. Then, the correction processor 114 performs correction processing on the identified specific character rectangles. Specifically, the correction processor 114 adds a margin (margin rectangle) of a predetermined size to the specific character rectangle. For example, the correction processor 114 adds a margin of the same size as the specific character rectangle. In the example shown in FIG. 6, the correction processor 114 adds a margin Ka of the same height h4 as the specific character rectangle K24 to the specific character rectangle K24. The correction processor 114 also adds the margin Ka in a direction (upward in this case) perpendicular to the arrangement direction of the multiple single-character rectangles K21 to K28 (the horizontal direction in FIG. 6). The correction processor 114 may also add a margin such that the height of the margin rectangle added to the specific character rectangle is less than the reference height H2 (=H1×F2). That is, the correction processing unit 114 may perform a correction to add a margin to the single-character rectangle so that the distance from the outline side of the character string rectangle K1 to the single-character rectangle after the correction is less than the reference height H2.

[0034] In this way, the correction processing unit 114 corrects the specific character rectangle K24 by adding the margin Ka to the specific character rectangle K24. Fig. 7 shows the specific character rectangle K24 after correction (hereinafter referred to as "specific character rectangle K24'").

[0035] The size (height) of the margin Ka is not limited to the same height h4 as the specific character rectangle K24, but may be, for example, the difference between the average height of the single-character rectangles K21 to K28 and the height h4 of the specific character rectangle K24. This allows the height of the specific character rectangle K24 plus the margin Ka to be adjusted to the average height of the single-character rectangles K21 to K28.

[0036] The recognition processing unit 115 performs character recognition processing (OCR processing) on ​​the character string. Specifically, the recognition processing unit 115 performs OCR processing on the character string using the specific character rectangles corrected by the correction processing unit 114. For example, the recognition processing unit 115 performs OCR processing based on the character string rectangle K1 extracted by the character string extraction processing unit 112, the multiple single-character rectangles K2 extracted by the single-character extraction processing unit 113, and the single-character rectangle K2 corrected by the correction processing unit 114 (specific character rectangle K24' in FIG. 7).

[0037] For example, the recognition processing unit 115 performs preprocessing (processing such as background removal, inversion, ruled line removal, seal removal, and italic correction) to improve the accuracy of OCR, and then performs existing OCR processing.

[0038] The output processing unit 116 outputs the OCR result (character recognition result). For example, the output processing unit 116 outputs the OCR result to the request source that has output the character recognition request for the input image.

[0039] Furthermore, the output processing unit 116 outputs the learning data to the learning device 2 (see FIG. 1). The control unit 11 generates the specific character rectangle corrected by the correction processing unit 114 as learning data to be used for machine learning. The output processing unit 116 outputs the learning data (teacher data) including the single-character rectangle K24 (",") corrected by the correction processing unit 114 and the corrected specific character rectangle K24' to the learning device 2.

[0040] The learning device 2 performs machine learning using the learning data generated by the image processing device 1, and generates a trained model.

[0041] Machine learning includes algorithms such as supervised learning using supervised data, unsupervised learning using unsupervised data, and reinforcement learning. Furthermore, to realize these methods, a method called "deep learning" is used, which learns to extract features themselves. In this embodiment, the learning device 2 has learning models based on the various algorithms described above. The learning device 2 performs machine learning using supervised data and unsupervised data as input data, and is capable of generating a trained model that performs character recognition processing.

[0042] The trained model can be applied to an image processing device 1. For example, as shown in Fig. 1, when an input image to be subjected to character recognition is input to the image processing device 1, the image processing device 1 uses the trained model to perform OCR processing on the input image and output the OCR result. The image processing device 1 is an example of an output device of the present disclosure.

[0043] The trained model may be downloaded to the image processing device 1 for use, or may be stored in a server (cloud server) and accessed from a user terminal via the Internet or the like for use. For example, when an arbitrary input image is input to the user terminal, the trained model outputs an optimal character recognition result. That is, the user terminal may perform OCR processing on the input image using the trained model generated by the learning device 2 and output the OCR result. The user terminal may also include a control unit that presents to the user the OCR result obtained by performing OCR processing on a character string using the specific character rectangle corrected in the image processing device 1. The user terminal is an example of an output device of the present disclosure.

[0044] [Character Recognition Processing] FIG. 8 is a flowchart showing an example of the procedure of the character recognition process executed in the image processing device 1.

[0045] The present disclosure can be understood as a character recognition method (image processing method of the present disclosure) that executes one or more steps included in the character recognition process. One or more steps included in the character recognition process described herein may be omitted as appropriate. The steps in the character recognition process may be executed in a different order as long as the same operational effect is achieved. While the following description uses an example in which the control unit 11 of the image processing device 1 executes each step in the character recognition process, in other embodiments, one or more processors may execute each step in the character recognition process in a distributed manner. Furthermore, when the control unit 11 acquires character image data from an external device, it is possible for the control unit 11 to execute the character recognition process in parallel for each piece of character image data.

[0046] <Step S1> In step S1, the control unit 11 determines whether character image data has been acquired. Specifically, the control unit 11 acquires character image data of a handwritten document (e.g., the receipt in FIG. 2) from an external device or the like. If the control unit 11 has acquired character image data (S1: Yes), the control unit 11 shifts the process to steps S21 and S22. The control unit 11 waits until the character image data has been acquired (S1: No).

[0047] <Step S21> In step S21, the control unit 11 extracts a character string rectangle corresponding to a character string consisting of a plurality of characters from the character image data. Specifically, the control unit 11 executes a document part recognition process in the input image to recognize a character string consisting of a plurality of characters, and extracts a character string rectangle K1 corresponding to the recognized character string. For example, in the input image shown in FIG. 2, the control unit 11 extracts a character string rectangle K1 corresponding to a handwritten character string ("￥99,600-").

[0048] <Step S22> In step S22, the control unit 11 extracts a plurality of single-character rectangles corresponding to a plurality of characters from the character image data. Specifically, the control unit 11 recognizes characters in the input image on a character-by-character basis, and extracts a single-character rectangle K2 corresponding to the recognized character. For example, as shown in Fig. 4, the control unit 11 extracts eight single-character rectangles K21 to K28 from a character string rectangle K1.

[0049] The control unit 11 executes the processes of steps S21 and S22 in parallel. In another embodiment, the control unit 11 may execute the process of step S22 after step S21. After steps S21 and S22, the control unit 11 shifts the process to step S3.

[0050] <Step S3> In step S3, the control unit 11 determines whether the single-character rectangles extracted in step S22 include a single-character rectangle to be corrected (specific character rectangle). Specifically, the control unit 11 determines whether the single-character rectangles include a specific character rectangle whose position and size satisfy predetermined conditions.

[0051] For example, the control unit 11 determines whether the multiple single-character rectangles include a single-character rectangle whose area is less than a predetermined area (reference area Mb) (first condition), and whether the multiple single-character rectangles include a single-character rectangle that is positioned at a position that is more than a predetermined distance (reference height H2) away from the outer edge (e.g., the top edge) of the character string rectangle K1 (second condition).

[0052] For example, the control unit 11 multiplies the average area Ma of the one-character rectangles K21 to K28 by a correction coefficient F1 (e.g., F1=0.3) and sets the result as the reference area Mb (=Ma×F1). In the example shown in Fig. 5, the control unit 11 determines that the area M4 of the one-character rectangle K24, among the multiple one-character rectangles K21 to K28, is less than the reference area Mb.

[0053] Furthermore, for example, the control unit 11 sets the value obtained by multiplying the height H1 of the character string rectangle K1 by a correction coefficient F2 (for example, F2=0.5) as the reference height H2 (=H1×F2). In the example shown in Fig. 5, among the multiple single-character rectangles K21 to K28, the height h4 of the single-character rectangle K24 from the top side of the character string rectangle K1 is equal to or greater than the reference height H2.

[0054] In the example shown in FIG. 5, the first and second conditions are satisfied, so the control unit 11 determines that the single-character rectangles K21 to K28 include the single-character rectangle to be corrected (specific character rectangle K24). If the control unit 11 determines that the single-character rectangles include the specific character rectangle (S3: Yes), it proceeds to step S4. On the other hand, if the control unit 11 determines that the single-character rectangles do not include the specific character rectangle (S3: No), it proceeds to step S5. For example, the control unit 11 determines that the single-character rectangles do not include a specific character rectangle if the single-character rectangles do not include a single-character rectangle whose area is less than a predetermined area (reference area Mb) (if the first condition is not satisfied) or if the single-character rectangles do not include a single-character rectangle that is located at a predetermined distance (reference height H2) or more from an outer edge (e.g., the top edge) of the character string rectangle K1 (if the second condition is not satisfied).

[0055] As another embodiment, the control unit 11 may adopt either the first condition or the second condition in step S3.

[0056] <Step S4> In step S4, the control unit 11 executes a correction process to add a margin to the specific character rectangle. Specifically, the control unit 11 adds a margin of a predetermined size to the specific character rectangle. For example, in the example shown in Fig. 6, the control unit 11 adds a margin Ka of the same height h4 as the specific character rectangle K24 above the specific character rectangle K24.

[0057] <Step S5> In step S5, the control unit 11 executes preprocessing for OCR processing. Specifically, the control unit 11 executes existing preprocessing such as background removal, inversion, ruled line removal, seal removal, italic correction, etc. For example, if the specific character rectangle is not included in the multiple single-character rectangles extracted in step S22 (S3: No), the control unit 11 executes OCR preprocessing based on the character string rectangle K1 extracted in step S21 and the multiple single-character rectangles K2 extracted in step S22. On the other hand, if a specific character rectangle is included in the multiple single-character rectangles extracted in step S22 (S3: Yes), the control unit 11 performs OCR preprocessing based on the character string rectangle K1 extracted in step S21, the multiple single-character rectangles K2 extracted in step S22, and the single-character rectangle K2 corrected in step S4 (specific character rectangle K24' with margin Ka added).

[0058] <Step S6> In step S6, the control unit 11 executes OCR processing. The control unit 11 executes existing OCR processing on the character string rectangle and single character rectangle after pre-processing of the OCR processing. After executing the OCR processing, the control unit 11 outputs the OCR result.

[0059] In this way, the control unit 11 executes the character recognition process. The control unit 11 also repeatedly executes the character recognition process every time character image data (input image) to be subjected to character recognition is acquired.

[0060] As described above, the image processing device 1 according to this embodiment acquires character image data, extracts character string rectangles corresponding to a character string consisting of a plurality of characters from the character image data, and extracts a plurality of single-character rectangles corresponding to each of the plurality of characters from the character image data. Furthermore, if the plurality of single-character rectangles includes a specific single-character rectangle whose position and size satisfy predetermined conditions, the image processing device 1 corrects the specific single-character rectangle and performs character recognition processing (OCR processing) on ​​the character string using the corrected specific single-character rectangle.

[0061] According to the above configuration, when a string of handwritten characters includes small handwritten characters such as commas, periods, and punctuation marks, the OCR process is performed by correcting the individual character rectangles of these small handwritten characters according to their position and size. This allows the position and size of small handwritten characters to be properly recognized, thereby improving the character recognition accuracy of strings including small handwritten characters.

[0062] [Other embodiments] In the above-described embodiment, an example in which the single-character rectangle K2 is placed below the character string rectangle K1 (see FIG. 4, etc.) has been shown, but the present disclosure is not limited to this. The OCR process can be performed in a similar manner even when the single-character rectangle K2 is placed above the character string rectangle K1. For example, FIG. 9 shows the handwritten characters for the date "'24 / 3 / 1" (e.g., "Publication Date" in FIG. 2). Note that "'24" is an abbreviation for "2024." The control unit 11 extracts the character string rectangle K1 and single-character rectangles K31 to K37 from the input image. As shown in FIG. 9, when the single-character rectangle is placed above the character string rectangle K1, the control unit 11 determines, in the second condition, whether the single-character rectangle K2 is located at a predetermined distance (reference height H2) or more from the bottom edge of the character string rectangle K1. If the single-character rectangle K31 satisfies the first condition and is located at a distance of at least the reference height H2 (=H1×0.5) from the bottom side of the character string rectangle K1 (if the second condition is satisfied), the control unit 11 adds a margin Ka below the single-character rectangle K31, as shown in Fig. 10. Then, the control unit 11 performs OCR processing based on the character string rectangle K1, the single-character rectangles K31 to K37, and the specific character rectangle K31' with the margin Ka added (see Fig. 11).

[0063] In this way, the control unit 11 may determine whether the single-character rectangle that satisfies the first condition is located above or below the character string rectangle K1, and if it is located below, determine the second condition based on the height (distance) from the top side of the character string rectangle K1 (see Figure 5), or if it is located above, determine the second condition based on the height (distance) from the bottom side of the character string rectangle K1 (see Figure 9).

[0064] In another embodiment, the control unit 11 may determine the second condition based on the height (distance) from the bottom side of the character string rectangle K1 when a single-character rectangle that satisfies the first condition is located below the character string rectangle K1, and may determine the second condition based on the height (distance) from the top side of the character string rectangle K1 when the single-character rectangle is located above the character string rectangle K1. In this case, the control unit 11 determines whether the height from the outer side of the character string rectangle K1 is less than the reference height H2 (= H1 × F2). The correction coefficient F2 is set to a value in the range of 0.1 to 0.5, for example.

[0065] In another embodiment, when multiple single-character rectangles K2 included in a character string rectangle K1 are arranged vertically, the control unit 11 determines whether the single-character rectangle that satisfies the first condition is located on the right or left side of the character string rectangle K1, and if it is located on the right side, determines the second condition based on the width (distance) from the left side of the character string rectangle K1, and if it is located on the left side, determines the second condition based on the width (distance) from the right side of the character string rectangle K1.

[0066] In another embodiment, when a plurality of single-character rectangles includes a specific single-character rectangle whose position or size satisfies a predetermined condition, the image processing device 1 may correct the specific single-character rectangle and perform character recognition processing (OCR processing) on ​​the character string using the corrected specific single-character rectangle. Specifically, when a plurality of single-character rectangles includes a single-character rectangle located at a position that is a predetermined distance or more away from the outer edge of the character string rectangle K1 (when the second condition is satisfied), the image processing device 1 may identify the single-character rectangle as a specific character rectangle and correct the specific character rectangle. In another embodiment, when a plurality of single-character rectangles includes a single-character rectangle whose area is less than a predetermined area (when the first condition is satisfied), the image processing device 1 may identify the single-character rectangle as a specific character rectangle and correct the specific character rectangle. In other words, the image processing device 1 according to the present disclosure may be configured to identify and correct a specific character rectangle when at least one of the first condition and the second condition is satisfied.

[0067] In this embodiment, the specific character corresponding to the specific character rectangle is, for example, at least one of a comma, a period, an apostrophe, a quotation mark, a punctuation mark, a small letter, a subscript, and a superscript. According to the configuration of the present disclosure, these specific characters can be correctly recognized. Furthermore, the specific character may be a lowercase alphabetic character. For example, if a string contains a lowercase "c," according to the configuration of the present disclosure, the character "c" can be correctly recognized as a lowercase "c" without being erroneously recognized as an uppercase letter. Furthermore, the specific character is not limited to Japanese or alphabetic characters, but may be a character of another language.

[0068] In the image processing system 10, the image processing device 1 and the learning device 2 may be configured as an integrated device. Furthermore, the processing units of the image processing device 1 (the acquisition processing unit 111, the character string extraction processing unit 112, the single character extraction processing unit 113, the correction processing unit 114, the recognition processing unit 115, and the output processing unit 116) may be distributed and arranged in multiple devices.

[0069] [Disclosure Note] The following is a summary of the disclosure extracted from the above-described embodiment. Note that the configurations and processing functions described in the following supplementary notes can be selected and combined as desired.

[0070] <Appendix 1> an acquisition processing unit that acquires character image data; a first extraction processing unit that extracts a character string rectangle corresponding to a character string composed of a plurality of characters from the character image data; a second extraction processing unit that extracts a plurality of single-character rectangles corresponding to the plurality of characters from the character image data; a correction processing unit that corrects a specific character rectangle when the specific character rectangle includes a specific character rectangle whose position and / or size satisfy a predetermined condition; a recognition processing unit that executes character recognition processing of the character string using the specific single-character rectangle corrected by the correction processing unit; An image processing device comprising:

[0071] <Appendix 2> the correction processing unit adds a margin of a predetermined size to the specific single-character rectangle. 2. The image processing device according to claim 1.

[0072] <Appendix 3> the correction processing unit adds the margin having the same size as the size of the specific one-character rectangle. 3. The image processing device according to claim 2.

[0073] <Appendix 4> the correction processing unit adds the margin corresponding to the difference between the average height of the plurality of single-character rectangles and the height of the specific single-character rectangle. 3. The image processing device according to claim 2.

[0074] <Appendix 5> the correction processing unit adds the margin in a direction perpendicular to an arrangement direction of the plurality of single-character rectangles. 5. The image processing device according to any one of Supplementary Notes 2 to 4.

[0075] <Appendix 6> the correction processing unit corrects one of the single-character rectangles whose area is less than a predetermined area and which is located at a position that is a predetermined distance or more away from an outer edge of the character string rectangle, among the plurality of single-character rectangles; 6. An image processing device according to any one of appendices 1 to 5.

[0076] <Appendix 7> the correction processing unit performs correction by adding a margin to the one-character rectangle so that the distance from the outline side to the one-character rectangle after correction becomes less than the predetermined distance. 7. The image processing device according to claim 6.

[0077] <Appendix 8> The specific single character rectangle is at least one of a comma, a period, an apostrophe, a quotation mark, a punctuation mark, and a small character. 8. An image processing device according to any one of appendices 1 to 7.

[0078] <Appendix 9> generating the specific single-character rectangle corrected by the correction processing unit as learning data to be used in machine learning; 9. An image processing device according to any one of appendices 1 to 8.

[0079] <Appendix 10> an image processing device according to Supplementary Note 9; a learning device that generates a trained model by performing machine learning using the learning data generated by the image processing device; and An image processing system comprising:

[0080] <Appendix 11> An output device that performs character recognition processing on an input image using the trained model generated by the learning device described in Supplementary Note 10, and outputs the character recognition result.

[0081] <Appendix 12> An output device comprising a control unit that presents to a user the character recognition results obtained by performing character recognition processing on the character string using the specific single-character rectangle corrected in an image processing device described in any one of Appendices 1 to 9. [Explanation of symbols]

[0082] 1: Image processing device 2: Learning device 10: Image processing system 11: Control section 12: Storage section 13: Operation display section 14: Communications Department 111: Acquisition processing unit 112: String extraction processing unit 113: Single character extraction processing unit 114: Correction processing unit 115: Recognition processing unit 116: Output processing section

Claims

1. an acquisition processing unit that acquires character image data; a first extraction processing unit that extracts a character string rectangle corresponding to a character string composed of a plurality of characters from the character image data; a second extraction processing unit that extracts a plurality of single-character rectangles corresponding to the plurality of characters from the character image data; a correction processing unit that corrects a specific character rectangle when the specific character rectangle includes a specific character rectangle whose position and / or size satisfy a predetermined condition; a recognition processing unit that executes character recognition processing of the character string using the specific single-character rectangle corrected by the correction processing unit; An image processing device comprising:

2. the correction processing unit adds a margin of a predetermined size to the specific single-character rectangle. The image processing device according to claim 1 .

3. the correction processing unit adds the margin of the same size as the size of the specific one-character rectangle. The image processing device according to claim 2 .

4. the correction processing unit adds the margin corresponding to the difference between the average height of the plurality of single-character rectangles and the height of the specific single-character rectangle. The image processing device according to claim 2 .

5. the correction processing unit adds the margin in a direction perpendicular to an arrangement direction of the plurality of single-character rectangles. The image processing device according to claim 2 .

6. the correction processing unit corrects one of the plurality of single-character rectangles whose area is less than a predetermined area and which is located at a position that is a predetermined distance or more away from an outer edge of the character string rectangle. The image processing device according to claim 1 .

7. the correction processing unit performs correction by adding a margin to the one-character rectangle so that the distance from the outline side to the one-character rectangle after correction becomes less than the predetermined distance. The image processing device according to claim 6 .

8. The specific single character rectangle is at least one of a comma, a period, an apostrophe, a quotation mark, a punctuation mark, and a small character. The image processing device according to claim 1 .

9. generating the specific single-character rectangle corrected by the correction processing unit as learning data to be used in machine learning; The image processing device according to any one of claims 1 to 8.

10. The image processing device according to claim 9 ; a learning device that generates a trained model by performing machine learning using the learning data generated by the image processing device; and An image processing system comprising:

11. An output device that performs character recognition processing on an input image using the trained model generated by the training device according to claim 10, and outputs a character recognition result.

12. An output device comprising a control unit that presents to a user the character recognition results obtained by performing character recognition processing on the character string using the specific single-character rectangle corrected in the image processing device described in any one of claims 1 to 8.

13. acquiring character image data; extracting a character string rectangle corresponding to a character string composed of a plurality of characters from the character image data; extracting a plurality of single-character rectangles corresponding to the plurality of characters from the character image data; If the plurality of single-character rectangles includes a specific single-character rectangle whose position and / or size satisfy a predetermined condition, correcting the specific single-character rectangle; performing character recognition processing on the character string using the corrected specific character rectangle; An image processing method executed by one or more processors.

14. acquiring character image data; extracting a character string rectangle corresponding to a character string composed of a plurality of characters from the character image data; extracting a plurality of single-character rectangles corresponding to the plurality of characters from the character image data; If the plurality of single-character rectangles includes a specific single-character rectangle whose position and / or size satisfy a predetermined condition, correcting the specific single-character rectangle; performing character recognition processing on the character string using the corrected specific character rectangle; An image processing program for causing one or more processors to execute the above.

Images