Image processing device and image processing method

Abstract

In the present invention, a character string area detection unit (13) detects a character string area in a specific form area of a document image. A correction content detection unit (14) detects a correction content character string written in a specific correction content entry area of the document image, and specifies whether or not a character has been deleted and whether or not a character has been added with respect to the character string in the character string area on the basis of the correction content character string. A character string extraction unit (15) extracts a character string for the form area from the character string area in accordance with the result of the determination by the correction content detection unit (14) as to whether or not a character has been deleted and whether or not a character has been added.

Description

Image processing apparatus and image processing method

[0001] The present invention relates to an image processing apparatus and an image processing method.

[0002] An image forming apparatus performs character recognition processing on a specific area in a document image of a certain document, and acquires text data of a character string described in the specific area (see, for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2023-137094

[0004] However, in the above-described image forming apparatus, when a correction object such as a strikethrough is added superimposed on characters in a document, the character string including the object (that is, the correction location) may not be appropriately converted into text data.

[0005] The present invention has been made in view of the above problems, and an object thereof is to obtain an image processing apparatus and an image processing method capable of appropriately obtaining a corrected character string even when a character string in a specific area is corrected with a correction object.

[0006] The image processing apparatus according to the present invention includes a character string area detection unit that detects a character string area in a specific form area in a document image, (a) detects a correction content character string described in a specific correction content entry area in the document image, and (b) based on the correction content character string, a correction content detection unit that specifies the presence or absence of character deletion and the presence or absence of character addition to the character string in the character string area, and according to the determination results of the presence or absence of the character deletion and the presence or absence of the character addition by the correction content detection unit, a character string extraction unit that extracts a character string for the form area from the character string area.

[0007] The image processing method according to the present invention comprises the steps of: (a) detecting a string area in a specific form area of ​​a document image; (b) detecting a correction string written in a specific correction content entry area of ​​the document image; and (c) determining whether or not characters have been deleted or added to the string in the string area based on the correction string; and extracting a string for the form area from the string area according to the determination result of whether or not characters have been deleted or added in the correction content determination step.

[0008] The image processing program according to the present invention causes a computer to function as the above-mentioned character string area detection unit, the above-mentioned correction content detection unit, and the above-mentioned character string extraction unit.

[0009] According to the present invention, an image processing apparatus and image processing method are obtained that can appropriately obtain a corrected string even when a correction object has been applied to a string in a specific area.

[0010] The above or other objects, features, and advantages of the present invention will become even more apparent from the following detailed description in conjunction with the accompanying drawings.

[0011] Figure 1 is a block diagram showing the configuration of an image processing apparatus according to an embodiment of the present invention. Figure 2 is a diagram showing an example of a form area and a correction content entry area. Figure 3 is a diagram illustrating the correction character count check based on the correction content string. Figure 4 is a flowchart illustrating the operation of the image processing apparatus shown in Figure 1.

[0012] Embodiments of the present invention will be described below with reference to the figures.

[0013] Figure 1 is a block diagram showing the configuration of an image processing apparatus according to an embodiment of the present invention. The image processing apparatus shown in Figure 1 is an information processing apparatus such as a personal computer or a server, or an electronic device such as a digital camera or an image forming apparatus (scanner, multifunction printer, etc.), and comprises a processing unit 1, a storage device 2, a communication device 3, a display device 4, an input device 5, an internal device 6, and the like.

[0014] The arithmetic processing unit 1 includes a computer and operates as various processing units by executing image processing programs on that computer. Specifically, the computer includes a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc., and operates as a predetermined processing unit by loading programs stored in the ROM or storage device 2 into the RAM and executing them with the CPU. The arithmetic processing unit 1 may also include an ASIC (Application Specific Integrated Circuit) that functions as a specific processing unit.

[0015] The storage device 2 is a non-volatile storage device such as flash memory, and stores the image processing program 2a and data (such as form data 2b) necessary for the processing described later. The image processing program 2a is stored, for example, on a non-temporary, computer-readable recording medium and installed from that recording medium to the storage device 2.

[0016] Form data 2b is data that indicates the position and size of a form area within a specific target image where a string of a specific item, such as handwritten text or printed text, is entered by the user.

[0017] Communication device 3 is a device that communicates data with external devices, such as a network interface or peripheral device interface. Display device 4 is a device that displays various information to the user, such as a display panel such as a liquid crystal display. Input device 5 is a device that detects user operations, such as a keyboard or touch panel.

[0018] The internal device 6 is a device that performs a predetermined function of the image processing device. For example, if the image processing device is an image forming apparatus, the internal device 6 may be an image reading device that optically reads an image from a document, or a printing device that prints an image onto printing paper.

[0019] In this configuration, the arithmetic processing unit 1 operates as the aforementioned processing units—the target image acquisition unit 11, the form area identification unit 12, the string area detection unit 13, the correction content detection unit 14, and the string extraction unit 15—based on the image processing program 2a.

[0020] The target image acquisition unit 11 acquires document images as target images (raster image data) from the storage device 2, communication device 3, internal device 6, etc., and stores them in RAM or the like.

[0021] For example, this document image is a read image of a document in which the string to be extracted is written in the form area defined by form data 2b.

[0022] The form area identification unit 12 identifies the form area (specifically, its position and size) in the document image based on the form data 2b. Here, the form area is a rectangular area in which the string to be extracted is written. The string to be extracted (i.e., the target of character recognition processing) may be printed characters or handwritten characters.

[0023] Figure 2 shows an example of a form area and a correction entry area.

[0024] The string area detection unit 13 detects string areas in the document image described above, for example, in the form area set as shown in Figure 2. For example, the string area detection unit 13 performs character recognition processing using a machine learning trainer to detect string areas using an existing object detection method, and detects character objects and correction objects described later within the string area.

[0025] Figure 3 illustrates the character count check based on the corrected text string.

[0026] The correction content detection unit 14 (a) detects a correction content string written in a specific correction content entry area in the document image, for example as shown in Figure 3, and (b) determines whether or not characters have been deleted or added to the string in the string area based on the correction content string. The correction content entry area may be set in a predetermined fixed position in the document image, or it may be set in a position adjacent to the corresponding form area.

[0027] The correction strings shown in Figure 3 are "delete 3 characters" and "add 3 characters." Because the correction strings contain the characters for "deletion" and "addition," it is determined that there is a character deletion and a character addition in the string area.

[0028] Note that the correction content entry area shown in Figure 2 does not contain any correction text, and therefore no character count check is performed.

[0029] The string extraction unit 15 extracts the string for the form area from the string area according to the determination result of the correction content detection unit 14 regarding whether or not characters have been deleted or added.

[0030] In this embodiment, the correction content detection unit 14 identifies the number of characters to be deleted based on the correction content string if there are character deletions, and identifies the number of characters to be added based on the correction content string if there are character additions.

[0031] If there is character deletion, the string extraction unit 15 detects the character deletion location (the correction object described above) in the string area, identifies the number of characters at the deletion location, and determines whether the number of deleted characters based on the correction content string matches the number of characters at the deletion location. If there is character addition, the string extraction unit 15 detects the character addition location in the string area, identifies the number of characters at the addition location, and determines whether the number of added characters based on the correction content string matches the number of characters at the addition location.

[0032] In the case shown in FIG. 3, the number of characters to be deleted from the number in the correction content string "Delete 3 characters" is specified as 3, and the number of characters to be added from the number in the correction content string "Add 3 characters" is specified as 3.

[0033] Also, among the string areas in the form area, the longest string area is regarded as the basic string area, and among the other string areas, the string area adjacent to the basic string area is regarded as the character addition location. And in the case shown in FIG. 3, the number of characters in the string "Central Ward" of the string area at the character addition location (that is, the number of characters at the addition location) is specified as 3 by character recognition processing or the like.

[0034] Regarding the number of characters at the deletion location, the string extraction unit 15 may delete the strikethrough (correction object) superimposed on the string from the character deletion location, and perform character recognition processing on the string after deleting the strikethrough to specify the number of characters at the deletion location. In the case shown in FIG. 3, the string "Tsurumi Ward" is obtained by character recognition processing from the correction location after deleting the strikethrough, and the number of characters at the deletion location is specified as 3.

[0035] Alternatively, the string extraction unit 15 may specify the number of characters at the deletion location based on the width of the characters in the string area (for example, the width of one character such as "City" in FIG. 3) and the width of the character deletion location (for example, the width of the deleted "Tsurumi Ward" part in FIG. 3). In the case shown in FIG. 3, since the correction location is three characters, the width of the character deletion location is about three times the width of the character, and the value obtained by dividing the width of the character deletion location by the width of the character (specifically, the integer value closest to that value, here 3) is used as the number of characters at the deletion location.

[0036] Further, when the string extraction unit 15 determines that the above-mentioned number of deleted characters does not match the number of characters at the character deletion location (the number of characters at the deletion location), and when it determines that the above-mentioned number of added characters does not match the number of characters at the character addition location (the number of characters at the addition location), the extraction of the string for the form area is prohibited because the correction in the document is not performed correctly or the correction in the document image is not recognized correctly.

[0037] On the other hand, if (a) it is determined that no characters have been added and the number of deleted characters matches the number of characters at the location where the characters were deleted, (b) it is determined that no characters have been deleted and the number of added characters matches the number of characters at the location where the characters were added, and (c) it is determined that the number of deleted characters matches the number of characters at the location where the characters were deleted, and the number of added characters matches the number of characters at the location where the characters were added, then the string extraction unit 15 extracts the string for the form area.

[0038] Specifically, when there are character deletions and character additions, the string extraction unit 15 obtains the string (text data) in the basic string area using character recognition processing, obtains the string at the character addition location using character recognition processing, and inserts the string at the character addition location into the character correction location in the string in the basic string area.

[0039] For example, in the case shown in Figure 3, the string "Osaka City Tamatsukuri" in the basic string area and the string "Chuo Ward" at the text addition location are obtained. Because there is a text correction location between "Osaka City" and "Tamatsukuri," the string obtained for the form area is "Osaka City Chuo Ward Tamatsukuri." Note that if there is no text addition, this type of text insertion will not occur.

[0040] Furthermore, the string extraction unit 15 informs the user of the determination result of whether the number of deleted characters matches the number of characters at the deletion location, and whether the number of added characters matches the number of characters at the addition location. For example, the string extraction unit 15 displays on the display device 4 whether or not characters were deleted, the number of deleted characters, the number of characters at the deletion location, and the determination result of whether or not the number of deleted characters matches the number of characters at the deletion location. It also displays on the display device 4 whether or not characters were added, the number of added characters, the number of characters at the addition location, and the determination result of whether or not the number of added characters matches the number of characters at the addition location.

[0041] In addition, when a specific operation mode is set by the user, even if there is character deletion, the character string extraction unit 15 may not detect the character deletion location in the character string area, may not specify the number of characters in the character deletion location, and may not determine whether the number of deleted characters based on the corrected content character string matches the number of characters in the character deletion location. In that case, when the character string extraction unit 15 determines that the number of additional characters does not match the number of characters in the character addition location, it prohibits the extraction of the character string for the form area, and when it determines that the number of additional characters matches the number of characters in the character addition location, it extracts the character string for the form area.

[0042] Next, the operation of the image processing apparatus shown in FIG. 1 will be described. FIG. 4 is a flowchart for explaining the operation of the image processing apparatus shown in FIG. 1.

[0043] When the target image acquisition unit 11 acquires a document image (step S1), the correction content detection unit 14 determines whether there is a corrected content character string in the correction content entry area in the document image (step S2).

[0044] When it is determined that there is no corrected content character string, the form area identification unit 12 identifies the form area in the document image where there is a character string to be extracted, the character string area detection unit 13 identifies the character string area in the form area, and the character string extraction unit 15 performs character recognition processing to extract the character string corresponding to the form area from the character string area (step S3).

[0045] On the other hand, the correction content detection unit 14 determines whether there is character deletion based on the corrected content character string (step S4).

[0046] When there is character deletion, the correction content detection unit 14 specifies the number of deleted characters based on the corrected content character string (step S5), the character string extraction unit 15 specifies the deletion location (correction object) in the character string area (step S6), and specifies the number of characters in the deletion location (step S7).

[0047] Also, the correction content detection unit 14 determines whether there is character addition based on the corrected content character string (step S8).

[0048] When there is character addition, the correction content detection unit 14 identifies the number of added characters based on the correction content character string (step S9), the character string extraction unit 15 identifies the addition location (adjacent character string area) in the character string area (step S10), and identifies the number of characters at the addition location (step S11).

[0049] Then, when there is character deletion, the character string extraction unit 15 determines whether the number of deleted characters matches the number of characters at the deletion location, and when there is character addition, determines whether the number of added characters matches the number of characters at the addition location (step S12).

[0050] When the numbers of characters for character deletion do not match each other or the numbers of characters for character addition do not match each other, the character string extraction unit 15, when there is character deletion, displays the number of deleted characters and the number of characters at the deletion location on the display device 4, and when there is character addition, displays the number of added characters and the number of characters at the addition location on the display device 4 (step S13). At this time, a message indicating that the numbers of added and / or deleted characters do not match may be displayed together. In this case, extraction of the character string corresponding to the form area is not performed.

[0051] On the other hand, when the numbers of characters in the case of character deletion match each other and the numbers of characters in the case of character addition match each other, the character string extraction unit 15 performs the character recognition process as described above to obtain the character string corresponding to the form area from the character string in the basic character string area and the character string in the adjacent character string area (step S3).

[0052] As described above, according to the above embodiment, the character string area detection unit 13 detects the character string area in a specific form area in the document image. The correction content detection unit 14 (a) detects the correction content character string described in a specific correction content entry area in the document image, and (b) based on the correction content character string, identifies the presence or absence of character deletion and the presence or absence of character addition with respect to the character string in the character string area. The character string extraction unit 15 extracts the character string for the form area from the character string area according to the determination results of the presence or absence of character deletion and the presence or absence of character addition by the correction content detection unit 14.

[0053] This ensures that even if corrections (such as strikethrough) have been made to the text within a given area, the corrected text is obtained correctly.

[0054] Furthermore, various changes and modifications to the embodiments described above will be obvious to those skilled in the art. Such changes and modifications may be made without deviating from the spirit and scope of the subject matter and without diminishing the intended advantages. In other words, such changes and modifications are intended to be included in the claims.

[0055] For example, in the above embodiment, the string extraction unit 15 may determine whether or not there is a seal impression in the correction content entry area, and if there is a seal impression in the correction content entry area, it may perform the above-described process, and if there is no seal impression in the correction content entry area, it may not perform the above-described process (including the extraction of the string from the string area) even if there is a correction content string.

[0056] Furthermore, in the above embodiment, if there are locations in the string area where characters need to be deleted even though there are no correction strings for character deletion, the string extraction unit 15 may refrain from extracting strings from the string area. Similarly, if there are locations in the form area where characters need to be added even though there are no correction strings for character addition, the string extraction unit 15 may refrain from extracting strings from the string area.

[0057] Furthermore, in the above embodiment, the image processing device is a device used directly by the user, but it may also be a device such as a server (e.g., a cloud server) used indirectly by the user via a network. In that case, the user interface of the client device is used instead of the display device 4 and the input device 5.

[0058] The present invention is applicable, for example, to an image processing apparatus.

Claims

1. An image processing apparatus comprising: a string area detection unit for detecting a string area in a specific form area of ​​a document image; a correction content detection unit for (a) detecting a correction content string written in a specific correction content entry area of ​​the document image; and (b) determining whether or not to delete characters and whether or not to add characters to the string in the string area based on the correction content string; and a string extraction unit for extracting a string for the form area from the string area according to the determination result of the correction content detection unit regarding whether or not to delete characters and whether or not to add characters.

2. The correction content detection unit, based on the correction content string, identifies the number of deleted characters based on the correction content string if there are character deletions, and identifies the number of added characters based on the correction content string if there are character additions. The string extraction unit, (a) if there are character deletions, detects the character deletion location in the string area, identifies the number of characters at the character deletion location, and determines whether the number of deleted characters based on the correction content string matches the number of characters at the character deletion location. If there are character additions, detects the character addition location in the string area, identifies the number of characters at the character addition location, and determines whether the number of added characters based on the correction content string matches the number of characters at the character addition location. (b) If it is determined that the number of deleted characters does not match the number of characters at the character deletion location. (b) If it is determined that the number of added characters does not match the number of characters at the location where the characters are added, the extraction of a string from the form area is prohibited. (b) If it is determined that there are no added characters and the number of deleted characters matches the number of characters at the location where the characters are deleted, if it is determined that there are no deleted characters and the number of added characters matches the number of characters at the location where the characters are added, and if it is determined that the number of deleted characters matches the number of characters at the location where the characters are deleted, the extraction of a string from the form area is performed. The image processing apparatus according to claim 1.

3. The image processing apparatus according to claim 2, wherein, when a specific operating mode is set, the string extraction unit does not determine whether the number of deleted characters based on the corrected string matches the number of characters at the character deletion location, (b1) if it determines that the number of added characters does not match the number of characters at the character addition location, it prohibits the extraction of the string for the form area, and (b2) if it determines that the number of added characters matches the number of characters at the character addition location, it performs the extraction of the string for the form area.

4. The image processing apparatus according to claim 2, characterized in that the string extraction unit removes the strikethrough line superimposed on the string from the character deletion location, and performs character recognition processing on the string after the strikethrough line has been removed to determine the number of characters in the deletion location.

5. The image processing apparatus according to claim 2, characterized in that the string extraction unit determines the number of characters to be deleted based on the width of the characters in the string area and the width of the character deletion location.

6. The image processing apparatus according to claim 2, characterized in that the string extraction unit informs the user of the determination result of whether the number of characters to be deleted matches the number of characters at the location where the characters were deleted, and whether the number of characters to be added matches the number of characters at the location where the characters were added.

7. An image processing method comprising: (a) detecting a string area in a specific form area in a document image; (b) detecting a correction string written in a specific correction area in the document image; and (c) determining whether or not characters have been deleted or added to the string in the string area based on the correction string; and (d) extracting a string for the form area from the string area according to the determination result of whether or not characters have been deleted or added in the correction content determination step.

Images