Information processing system, information processing method, and program

Abstract

To provide a system capable of appropriately extracting information from an image.SOLUTION: In an information processing system, an electronic data generating apparatus comprises: a first information acquisition unit that acquires, from a first optical character recognition apparatus which recognizes each of a plurality of characters included in a target image as first text information, the first text information for each of the plurality of characters and a first accuracy index indicating a degree of accuracy of recognition of the characters indicated by the first text information; a second information acquisition unit that acquires second text information from a second optical character recognition apparatus different from the first optical character recognition apparatus, which recognizes, when the first accuracy index indicating the degree of accuracy of recognition of the characters indicated by the first text information is determined to be lower than a first standard regarding the accuracy of recognition of characters included in the image, at least one character included in a specific range image, which is an image of a partial range of the target image including characters corresponding to the first accuracy index determined to be lower than the first standard, as the second text information; and a character identification unit that identifies the characters included in the target image based on the first text information and the second text information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] A system for extracting information from document images using a machine learning module has been disclosed (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2022-79439 Summary of the Invention [Problem to be solved by the invention]

[0004] The document data extraction system described in Patent Document 1 acquires image data associated with a document and extracts metadata from the image data using optical character recognition. The metadata specifies a string of text content items and text content item features associated with each text content item in the string of text content items. The document data extraction system uses a machine learning module to determine one or more text content items associated with a key based on the string of text content items and the text content item features. This allows the document data extraction system to extract information from the document image.

[0005] However, if metadata cannot be generated appropriately from image data using a single optical character recognition device that performs optical character recognition, a problem occurs in that information cannot be extracted appropriately from document images.

[0006] Therefore, in order to solve the above problems, an object of the present invention is to provide a system that can appropriately extract information from an image. [Means for solving the problem]

[0007] An information processing system according to one aspect of the present invention includes a target image acquisition unit that acquires a target image including characters from a predetermined device; a first information acquisition unit that acquires, from a first optical character recognition device that recognizes each of a plurality of characters included in the target image as first text information, the first text information for each of the plurality of characters and a first accuracy index that indicates the degree of accuracy of recognition for the character indicated by the first text information; and a first reference index that is based on at least one of the first accuracy indexes for the plurality of characters and that indicates the degree of accuracy of recognition of the character included in the image. If the first reference indicator is determined to be lower than the first standard, a specific range image is an image of a partial range of the target image that includes characters corresponding to the first reference indicator determined to be lower than the first standard, and the specific range image is recognized as second text information for at least one character included in the specific range image.A second information acquisition unit acquires the second text information about characters included in the specific range image from a second optical character recognition device different from the first optical character recognition device, and a character identification unit identifies characters included in the target image based on the first text information and the second text information.

[0008] An information processing method according to one embodiment of the present invention includes a computer executing the following steps: acquiring a target image including characters from a predetermined device; acquiring, from a first optical character recognition device that recognizes each of a plurality of characters included in the target image as first text information, first text information for each of the plurality of characters and a first accuracy index indicating the degree of recognition accuracy for the characters indicated by the first text information; when a first reference index based on at least one of the first accuracy indexes for the plurality of characters is determined to be lower than a first reference standard for the accuracy of recognition of characters included in an image, acquiring second text information for the characters included in the specific range image from a second optical character recognition device different from the first optical character recognition device that recognizes each of at least one character included in the specific range image as second text information; and identifying the characters included in the target image based on the first text information and the second text information.

[0009] A program according to one embodiment of the present invention causes a computer to execute the following steps: acquire a target image including characters from a specified device; acquire, from a first optical character recognition device that recognizes each of a plurality of characters included in the target image as first text information, first text information for each of the plurality of characters and a first accuracy index indicating the degree of recognition accuracy for the characters indicated by the first text information; when a first reference index based on at least one of the first accuracy indexes for the plurality of characters is determined to be lower than a first reference standard for the accuracy of recognition of characters included in an image, acquire second text information for characters included in the specific range image from a second optical character recognition device different from the first optical character recognition device that recognizes each of at least one character included in the specific range image as second text information; and identify characters included in the target image based on the first text information and the second text information. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide a system that can appropriately extract information from an image. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an overview of an electronic data generation system. [Figure 2] 1 is a database showing an example of target image information. [Figure 3] 10 is a database showing an example of specific range image information. [Figure 4] FIG. 10 is a diagram showing an example of a screen displayed on a display unit. [Figure 5] 10 is a flowchart showing a processing procedure of the electronic data generation system. [Figure 6]10 is a table showing validity indicators for line information, which is text information of one line of a character string. [Figure 7] FIG. 2 illustrates an example of a hardware configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0012] An electronic data generation system 10 according to one embodiment of the present invention will be described in detail below with reference to the drawings. However, the embodiment described below is merely an example, and is not intended to exclude various modifications or applications of techniques not explicitly described below. In other words, the present invention can be implemented by various modifications or combinations of the embodiments without departing from the spirit of the invention. Furthermore, in the following description of the drawings, identical or similar parts are denoted by identical or similar reference numerals.

[0013] In this embodiment, the terms "unit," "device," and "system" do not simply refer to physical means, but also include cases where the functions of the "unit," "device," or "system" are realized by software. Furthermore, the functions of one "unit," "device," or "system" may be realized by two or more physical means or devices, and the functions of two or more "units," "devices," or "systems" may be realized by one physical means or device. Furthermore, the various functions described below of each of the multiple devices constituting the electronic data generation system 10 may be configured to be executed by another device in the multiple devices.

[0014] ===Outline of Electronic Data Generation System 10=== <<Configuration Overview>> An overview of the electronic data generation system 10 will be described with reference to Fig. 1. Fig. 1 is a diagram showing an overview of the electronic data generation system 10.

[0015] The electronic data generation system 10 is a system that uses multiple optical character recognition devices to accurately generate digital documents from images. Specifically, the electronic data generation system 10 generates text information, which is digital data, from images specified in graphic formats such as JPEG (Joint Photographic Experts Group), TIFF (Tagged Image File Format), and PNG (Portable Network Graphics) or images in PDF (Portable Document Format) data (hereinafter referred to as "target images") through the multiple optical character recognition devices.

[0016] The target image is, for example, an image of various contracts, papers, etc. For convenience, the following description will be given assuming that the target image is an image of a contract page as an example.

[0017] The electronic data generation system 10 includes, for example, an electronic data generation device 100, a first optical character recognition device 200, a second optical character recognition device 300, and a user terminal 400.

[0018] The electronic data generation device 100 is a device that outputs a digital document that is the result of character recognition of a target image, based on the results of character recognition of the target image by each of two different optical character recognition devices.

[0019] The first optical character recognition device 200 is a device that performs character recognition on a target image.

[0020] The second optical character recognition device 300 is a device that performs character recognition on an image within a predetermined range of the target image, which is acquired from the electronic data generation device 100.

[0021] The electronic data generation device 100, the first optical character recognition device 200, and the second optical character recognition device 300 may be, for example, a cloud computer, a server computer, a personal computer (e.g., a desktop, laptop, tablet, etc.), a media computer platform (e.g., a cable or satellite set-top box, digital video recorder), a handheld computer device (e.g., a PDA, email client, etc.), or other types of computers or communication platforms. Note that at least a portion of the processing in the electronic data generation device 100, the first optical character recognition device 200, and the second optical character recognition device 300 may be implemented by one or more computers (for example, and not by way of limitation, cloud computing configured by one or more computers).

[0022] The user terminal 400 is a device that receives user input and displays various types of information.

[0023] The user terminal 400 may be, for example, a smartphone, a mobile phone (feature phone), a personal computer (e.g., a desktop, laptop, tablet, etc.), a media computing platform (e.g., a cable or satellite set-top box, a digital video recorder), a handheld computing device (e.g., a personal digital assistant (PDA), an email client, etc.), a wearable device (e.g., a glasses-type device, a watch-type device, etc.), or another type of computer or communication platform.

[0024] <<Processing Overview>> An overview of the processing of the electronic data generation system 10 will be described with reference to FIG.

[0025] First, in step S10, the electronic data generation device 100 transmits a target image acquired from a predetermined device to the first optical character recognition device 200.

[0026] In step S11, the first optical character recognition device 200 recognizes characters included in a target image (e.g., an image of one page) and generates text information about the recognized characters (hereinafter referred to as "first text information"). The first text information may include the coordinates of the characters on the target image. At this time, the first optical character recognition device 200 generates a degree of recognition accuracy for the characters indicated by the generated first text information (hereinafter referred to as "first accuracy index"). For convenience, the first text information, the first accuracy index, and the coordinates may be collectively referred to as "first generated information" below.

[0027] The first optical character recognition device 200 transmits the first generated information to the electronic data generation device 100 .

[0028] In step S12, if the electronic data generation device 100 determines that a reference index (hereinafter referred to as the "first reference index") based on at least one of the first accuracy indexes for multiple characters is lower than a standard (hereinafter referred to as the "first standard") regarding the accuracy of recognizing characters contained in an image, it identifies an image (hereinafter referred to as the "specific range image") in a predetermined range of the target image that contains characters corresponding to the reference index.

[0029] The first reference index may be, for example, a first accuracy index for each of a plurality of characters, or an average value of the first accuracy indexes for a plurality of characters included in a line-unit image or a block-unit image.

[0030] The first criterion is, for example, a threshold value comparable to the first accuracy index.

[0031] The specific range image may be, for example, an image of a single sentence (hereinafter referred to as a "block image") obtained by segmenting a portion of a target image in one page unit, an image of one line (hereinafter referred to as a "line image"), or a character image which is an image of one character. For convenience, the specific range image will be described below as a "line image."

[0032] The electronic data generation device 100 transmits the specific range image to the second optical character recognition device 300. That is, the electronic data generation device 100 causes an optical character recognition device different from the optical character recognition device that performed character recognition on the target image to perform character recognition again on, for example, a specific range image (e.g., a line image) of a partial range of the target image, including characters determined to have low accuracy of character recognition.

[0033] In step S13, the second optical character recognition device 300 recognizes characters included in the specific range image and generates text information about the recognized characters (hereinafter referred to as "second text information"). The second text information may include the coordinates of the characters on the target image. At this time, the second optical character recognition device 300 generates a degree of recognition accuracy for the characters indicated by the generated second text information (hereinafter referred to as "second accuracy index"). Note that, hereinafter, the second text information, second accuracy index, and coordinates may be collectively referred to as "second generated information."

[0034] The second optical character recognition device 300 transmits the second generated information to the electronic data generation device 100 .

[0035] In step S14, the electronic data generation device 100 identifies the characters contained in the specific range image based on the comparison result (hereinafter referred to as the "first comparison result") between a reference index (hereinafter referred to as the "second reference index") based on at least one of the second accuracy indexes for each of at least one character contained in the specific range image and a standard (hereinafter referred to as the "second standard") regarding the accuracy of recognizing the characters contained in the image.

[0036] The second reference index may be, for example, the second accuracy index for each character included in the specific range image, or the average value of the second accuracy indexes for multiple characters included in the specific range image.

[0037] The second criterion is, for example, a threshold value that can be compared with the second accuracy index.

[0038] The electronic data generation device 100 transmits to the user terminal 400 a screen including information indicating the characters included in the identified specific range image (hereinafter referred to as the "character recognition result").

[0039] As described above, when the accuracy of character recognition by the first optical character recognition device 200 for characters contained in a target image on a page basis is low, the electronic data generation device 100 obtains the results of character recognition of a specific range image (here, as an example, a line image on a line basis) of a part of the target image including the characters using a second optical character recognition device 300 different from the first optical character recognition device 200, and identifies the characters contained in the target image based on the results.

[0040] In other words, the electronic data generation device 100 uses multiple optical character recognition devices with different character recognition target ranges (for example, the first optical character recognition device 200 performs character recognition on a page-by-page basis, and the second optical character recognition device 300 performs character recognition on a line-by-line basis) to perform character recognition on images with low character recognition accuracy, thereby making it possible to achieve highly accurate character recognition.

[0041] Furthermore, in the electronic data generation system 10, rather than performing all character recognition in the target range using the second optical character recognition device 300, which has a higher cost of performing character recognition than the first optical character recognition device 200, the accuracy of character recognition is lower in the first optical character recognition device 200, and character recognition is performed in a narrower range than the target range, thereby making it possible to increase the accuracy of character recognition and reduce the cost of character recognition.

[0042] The specific range image sent to the second optical character recognition device 300 is not limited to a line image, but may be an image of a page, a block image, or a character image.

[0043] Furthermore, the electronic data generation system 10 may further include at least one optical character recognition device in addition to the first optical character recognition device 200 and the second optical character recognition device 300. In this case, the optical character recognition device may be one that performs character recognition on images in the same range as either the first optical character recognition device 200 or the second optical character recognition device 300, or one that performs character recognition on images in a narrower range than the range of character recognition performed by the first optical character recognition device 200 and the second optical character recognition device 300. This enables the electronic data generation system 10 to achieve more accurate character recognition.

[0044] Electronic Data Generation Device 100 As shown in FIG. 1, the electronic data generation device 100 includes a memory unit 101, a target image acquisition unit 102, a first information acquisition unit 103, a first judgment unit 104, a specific range identification unit 105, an information transmission unit 106, a second information acquisition unit 107, a second judgment unit 108, a character identification unit 109, and a display processing unit 110.

[0045] The storage unit 101 includes, for example, target image information D101a and specific range image information D101b.

[0046] The target image information D101a will be described with reference to Fig. 2. Fig. 2 is a database showing an example of the target image information 101a. The target image information 101a is a database in which data relating to the target image is stored.

[0047] As shown in FIG. 2, the target image information D101a includes items such as [Target Image ID], [Target Image], [Block Information], [Line Information], [Character Information], [Coordinates], and [First Accuracy Index]. [Target Image ID] stores identification information that can uniquely identify the target image. [Target Image] stores the target image. [Block Information] stores text information of a block image that indicates a block of text in the target image (hereinafter referred to as "block information"). [Line Information] stores text information for each line included in the block image (hereinafter referred to as "line information"). [Character Information] stores text information of characters included in each line (hereinafter referred to as "character information"). [Coordinates] stores the coordinates in the target image of the characters (character information) indicated by each piece of first text information. [First Accuracy Index] stores a first accuracy index that indicates the degree of accuracy of the characters indicated by each piece of first text information.

[0048] The specific range image information D101b will be described with reference to Fig. 3. Fig. 3 is a database showing an example of the specific range image information 101b. The specific range image information 101b is a database in which data relating to specific range images is stored.

[0049] As shown in FIG. 3, the specific range image information D101b includes items such as a [specific range image ID], a [specific range image], a [second text information], a [coordinates], and a [second accuracy index]. The [specific range image ID] stores identification information that can uniquely identify the specific range image. The [specific range image] stores the specific range image. The [second text information] stores second text information of the character image included in the specific range image (the line image in FIG. 3). The [coordinates] stores the coordinates in the target image or the coordinates in the specific range image of the characters indicated by the second text information. The [second accuracy index] stores a second accuracy index that indicates the degree of accuracy of the characters indicated by the second text information.

[0050] The target image acquisition unit 102 acquires a target image from a predetermined device. The predetermined device may be, for example, a scanner device capable of acquiring an image of characters printed on paper, or a server device that stores the target image. The target image acquisition unit 102 may transmit the acquired target image to the first optical character recognition device 200.

[0051] 1, the first information acquisition unit 103 acquires first generation information about the target image from the first optical character recognition device 200. Specifically, the electronic data generation device 100 acquires, for example, first text information (including coordinates) and a first accuracy index about each character image included in the target image from the first optical character recognition device 200. The first generation information is stored in the target image information D101a.

[0052] The first determination unit 104 determines whether the first reference index is lower than the first reference. Specifically, when the threshold value as the first reference is "0.6" and the image in a predetermined range of the target image (for example, a line image) is "ABC," and the first accuracy indexes of the first text information "A," "B," and "D" (character image "C" is recognized as "D") are "0.99," "0.99," and "0.55," respectively, the first determination unit 104 determines that the first reference index (here, the first accuracy index "0.55" corresponding to "D") in the image in the predetermined range is lower than the first reference (here, the threshold value "0.6").

[0053] In other words, if at least one (or the average value) of the first accuracy indicators corresponding to the first text information in a predetermined range of images (e.g., block images, line images, or character images) included in the target image is lower than a threshold value, the electronic data generation device 100 may determine that accurate character recognition has not been possible for the images in the predetermined range.

[0054] The specific range identifying unit 105 identifies a specific range image, which is an image of a partial range of the target image, including characters corresponding to the first reference indicator determined to be lower than the first standard based on the determination result (hereinafter referred to as the "first determination result") by the first determining unit 104. In this case, the specific range identifying unit 105 identifies the coordinates of each character image included in the target image acquired from the first optical character recognition device 200, and identifies a specific range image (e.g., a line image) including characters corresponding to the first reference indicator based on the coordinates.

[0055] Here, an overview of an example of a process for identifying a line image (hereinafter referred to as "line identification process") will be described. In the line identification process, first, a black dot image at the left edge of the target image is identified. Next, in the line identification process, a histogram (for example, the horizontal axis represents the distance from the left edge of the target image and the vertical axis represents the number of dot images) is generated while identifying black dots in a predetermined width in the height direction from the identified dot image in the horizontal direction to the right. Next, a histogram is generated while similarly identifying black dot images in a predetermined width at an angle shifted from the horizontal. Then, in the line identification process, the coordinates of the upper left dot and the lower right dot of the line image are identified based on the histogram, thereby identifying the coordinates of the range of the line image. This makes it possible to appropriately identify a predetermined group of images containing multiple characters.

[0056] In addition, when the distance between the first distance range in which black dot images are identified and the second distance range in which black dot images are identified exceeds a predetermined distance in the histogram, the range in which black dot images are identified in the first distance range is identified as a first row image, and the range in which black dot images are identified in the second distance range is identified as a second row image different from the first row image. This makes it possible to identify row images that are in the same row but included in different block images as different rows.

[0057] The information transmitting unit 106 transmits the identified specific range image (for example, a line image) to the second optical character recognition device 300.

[0058] The second information acquisition unit 107 acquires second generation information about characters included in the specific range image from the second optical character recognition device 300. Specifically, the electronic data generation device 100 acquires, for example, second text information (including coordinates) and second accuracy indices (for example, "A: 0.99," "B: 0.98," and "C: 0.99") about each of the character images (for example, "A," "B," and "C") included in a line image (for example, "ABC") that is the specific range image from the second optical character recognition device 200.

[0059] The second determination unit 108 determines whether the second reference index is greater than or equal to the second reference. Specifically, the second determination unit 108 determines, for example, whether all of the second accuracy indices for the characters included in the specific range image are equal to or greater than a predetermined threshold. For example, if the predetermined threshold is "0.6" and the specific range image (e.g., a line image) is "ABC," and the second accuracy indices for the second text information "A," "B," and "C" are "0.99," "0.99," and "0.98," respectively, the second determination unit 108 determines that the second reference index is equal to or greater than the second reference. Note that the second determination unit 108 may also determine whether the average value of the second accuracy indices for the characters included in the specific range image is equal to or greater than a predetermined threshold.

[0060] The second determination unit 108 may determine that the second reference index is lower than the second reference index when any one of the second accuracy indices for the specific range image is lower than a predetermined threshold. For example, when the specific range image (e.g., a line image) is "ABC," if the second accuracy indices for the second text information "A," "B," and "C" are "0.99," "0.99," and "0.50," respectively, the second determination unit 108 determines that the second reference index is lower than the second reference index.

[0061] In this way, the electronic data generation device 100 determines that the second text information is able to recognize characters more accurately for character images in the target image than the first text information, for example, when all of the second accuracy indices for a specific range image exceed a predetermined threshold.

[0062] The character identification unit 109 identifies characters included in the specific range image (i.e., the target image) based on the determination result (hereinafter referred to as the "second determination result") by the second determination unit 108. Specifically, when the second reference index is equal to or greater than the second reference, the character identification unit 109 identifies characters indicated by the second text information corresponding to the second accuracy index as characters included in the specific range image.

[0063] For example, if the second accuracy index (second reference index) of the second text information "A", "B", and "C" are "0.99", "0.99", and "0.98", respectively, the character identification unit 109 identifies "ABC" indicated by the second text information as a character string in the specific range image because the second reference index is greater than or equal to the second reference (for example, a threshold value of "0.6").

[0064] This makes it possible for a character string within a predetermined range that includes characters with low accuracy in the first character recognition by the first optical character recognition device 200 to be properly recognized in the second character recognition by the second optical character recognition device 300.

[0065] On the other hand, when the second reference index is lower than the second reference, the character identification unit 109 identifies, based on the result of determining the magnitude relationship between the first accuracy index and the second accuracy index, either the character indicated by the first text information corresponding to the first accuracy index or the character indicated by the second text information corresponding to the second accuracy index as the character in the specific range image.

[0066] Specifically, the character identification unit 109 identifies the minimum value that indicates the higher of the minimum value of the second accuracy index of each character image included in the specific range image acquired from the second optical character recognition device 300 and the minimum value of the first accuracy index corresponding to the first text information of the image corresponding to the specific range image acquired from the first optical character recognition device 200. The character identification unit 109 identifies the character indicated by the text information (the first text information or the second text information) corresponding to the accuracy index indicating the identified minimum value as the character included in the specific range image.

[0067] For example, for the line image "ABC," if the first accuracy indices for the first text information "A," "F," and "D" (here, character image "B" is recognized as "F" and "C" is recognized as "D") are "0.99," "0.40," and "0.55," and the second accuracy indices for the second text information "A," "B," and "E" (character image "C" is recognized as "E") are "0.99," "0.99," and "0.50," respectively, the character identification unit 109 determines that the lowest value of the second accuracy indices, "0.50," is higher than the lowest value of the first accuracy indices, "0.40." In this case, the character identification unit 109 identifies "ABE" indicated by the second text information as the character string in the specific range image.

[0068] This makes it possible to adopt a character string that is estimated to have been recognized more accurately from a predetermined range of character strings that includes characters with low accuracy in the first character recognition by the first optical character recognition device 200 and a character string in the second character recognition by the second optical character recognition device 300.

[0069] In the above description, the minimum value indicating the higher of the minimum value of the second accuracy index and the minimum value of the first accuracy index is identified, but this is not limited to this. For example, the character identification unit 109 may use an average value instead of the minimum value. In this case, the character indicated by the text information corresponding to the accuracy index indicating the higher average value (second reference index) may be identified as the character of the specific range image. For example, the electronic data generation device 100 can reduce the user's effort in correction by performing a process of identifying the minimum value indicating the higher value among the minimum values, while performing a process of identifying the higher average value among the average values ​​can reduce the user's effort in correction, since it is possible to adopt text that is well-organized overall.

[0070] As described above, when the accuracy of character recognition by the first optical character recognition device 200 for an image in a predetermined range (for example, line by line) of the target image is low, the electronic data generation device 100 obtains the result of character recognition of the image in the predetermined range by the second optical character recognition device 300, which is different from the first optical character recognition device 200. In other words, by performing character recognition using two different optical character recognition devices, the electronic data generation device 100 can achieve highly accurate character recognition for images with low character recognition accuracy.

[0071] More specifically, in the electronic data generation system 10, it is desirable to perform character recognition over a wide range (for example, in units of a page) using a first optical character recognition device 200 that is inexpensive to perform character recognition and has low character recognition accuracy, and to perform character recognition over a narrower range using a second optical character recognition device 300 that is expensive to perform character recognition and has higher character recognition accuracy (for example, higher accuracy in character recognition on a line-by-line basis) than the first optical character recognition device 200. This makes it possible for the electronic data generation system 10 to increase the accuracy of character recognition and reduce the cost of character recognition.

[0072] The display processing unit 110 associates the target image with the text information (character recognition result) identified by the character identification unit 109 and displays them on the screen. An example screen will be described below with reference to Fig. 4. Fig. 4 is a diagram showing an example screen displayed on the display unit.

[0073] As shown in FIG. 4, the screen T10 includes a first display area T11 and a second display area T12. The first display area T11 is an area in which the target image is displayed. The first display area T11 is, for example, half of one side of the screen. The second display area T12 is an area in which first text information in the target image and second text information (line information in FIG. 4) in the specific range image are displayed. The second display area T12 is, for example, half of the other side of the screen.

[0074] The display processing unit 110, for example, identifiably displays characters included in the target image displayed in the first display area T11 and indicated by the line information displayed in the second display area. Specifically, as shown in FIG. 4, the display processing unit 110, for example, displays an object OT1 that identifies a line image of the target image, and displays an object OT2 in the line information corresponding to the line image. For example, the display color of the object OT1 is the same as the display color of the object OT2. This allows the electronic data generation device 100 to provide the user with a correspondence between the target image and the first text information and second text information that are the results of character recognition of a predetermined range of the target image, thereby enabling the user to easily understand any misrecognition of the target image.

[0075] <<Modifications>> The character identification unit 109 may identify either the characters indicated by the first text information or the characters indicated by the second text information as characters included in the target image based on the number of inputs, which is the number of times a specific range image is input to the second optical character recognition device 300. Specifically, for example, when the number of inputs of a specific range image (e.g., a line image) input to the second optical character recognition device 300 exceeds a predetermined number, the character identification unit 109 identifies the characters indicated by the first text information corresponding to the first accuracy index as characters included in the target image. As a result, when the cost of character recognition processing by the second optical character recognition device 300 is higher than the cost of character recognition processing by the first optical character recognition device 200, for example, or when the cost exceeds a certain cost, the electronic data generation system 10 can achieve cost reduction by performing character recognition using a less expensive optical character recognition device.

[0076] In this case, the display processing unit 110 may display the number of inputs, which is the number of times the specific range image has been input to the second optical character recognition device 300, at a predetermined position on the screen T10. Specifically, the display processing unit 110 may display it at a predetermined position in the second display area. Furthermore, as shown in FIG. 4, the display processing unit 110 may display the number of inputs in association with the second text information corresponding to the line image ("server" in FIG. 4) that has become the target of input to the second optical character recognition device 300 after the input number has been exceeded ("5 times" in FIG. 4). In this way, the electronic data generation system 10 allows the user to easily understand the specific range images for which character recognition by the second optical character recognition device 300 could not be performed due to the limited number of inputs.

[0077] ===First Optical Character Recognition Device 200=== 1, we will explain the configuration of the first optical character recognition device 200. For example, when a target image is input, the first optical character recognition device 200 is a device that recognizes characters included in the target image and generates, for example, first text information, a first accuracy index, and coordinates (first generated information) for each recognized character.

[0078] As shown in FIG. 1 , the first optical character recognition device 200 includes, for example, a storage unit 210, a transmission / reception unit 220, and a processing unit 230. The storage unit 210 stores various information. The processing unit 230 executes processing for character recognition. The transmission / reception unit 220 transmits and receives various information to and from the electronic data generation device 100. The processing unit 230 analyzes images using, for example, a neural network trained to distinguish between characters. The neural network includes, for example, multiple convolutional network layers and recurrent network layers. The processing unit 230 segments, for example, a target image into pages, blocks, lines, or characters. Character recognition is performed on characters included in the segmented images to generate, for example, first generated information for each character. The processing unit 230 may generate, for example, the first generated information for a group of segmented images (for example, a target image, a block image, or a line image).

[0079] ===Second Optical Character Recognition Device 300=== The configuration of the second optical character recognition device 300 will be described with reference to Fig. 1. The second optical character recognition device 300 is a device that, when a specific range image is input, recognizes characters included in the specific range image and generates, for example, second text information, a second accuracy index, and coordinates (second generated information) for each recognized character.

[0080] For example, when the specific range image is a line image, the second optical character recognition device 300 is preferably a device that can achieve higher accuracy in character identification than the first optical character recognition device 200. In this case, in the electronic data generation system 10, by using the second optical character recognition device 300, which has high accuracy in character recognition for line images, for line images for which the first optical character recognition device 200 has low accuracy in character recognition, it is possible to improve the accuracy of character recognition.

[0081] As shown in FIG. 1 , the second optical character recognition device 300 includes, for example, a storage unit 310, a transmission / reception unit 320, and a processing unit 330. The storage unit 310 stores various information. The processing unit 330 performs processing for character recognition. The transmission / reception unit 320 transmits and receives various information to and from the electronic data generation device 100. The processing unit 330 analyzes images using, for example, a neural network trained to distinguish between characters. The neural network includes, for example, multiple convolutional network layers and recurrent network layers. The processing unit 330 may be the same as the processing unit 230 of the first optical character recognition device 200, or may be a functional unit that performs processing specialized for character recognition of line images. In this case, the processing unit 330 segments the line image into characters, for example. Then, the processing unit 330 performs character recognition on the characters to generate, for example, second generated information for each character.

[0082] ===User terminal 400=== The configuration of the user terminal 400 will be described with reference to Fig. 1. As shown in Fig. 1, the user terminal 400 includes functional units such as a storage unit 410, a transmission / reception unit 420, and a display processing unit 430. Each functional unit is a function realized by, for example, a processor 1001 reading out a program stored in a memory 1002.

[0083] The storage unit 410 stores various types of information. The transmission / reception unit 420 transmits and receives various types of information to and from the electronic data generation device 100. The various types of information acquired by the transmission / reception unit 420 is stored in the storage unit 410. The display processing unit 430 displays a screen T10 acquired from the electronic data generation device 100 on the display unit.

[0084] ===Processing Procedure=== The processing procedure of the electronic data generation system 10 will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a flowchart showing the processing procedure of the electronic data generation system 10. Fig. 6 is a table showing validity indices for line information, which is text information of a line of character string. As an example, character recognition for "100BASE-TX switch set", which is a line of character string included in a target image, will be described below.

[0085] In step S100, the electronic data generation device 100 acquires a target image from a predetermined device. The electronic data generation device 100 stores the target image in the storage unit 101. The electronic data generation device 100 transmits the target image to the first optical character recognition device 200.

[0086] In step S101, the first optical character recognition device 200 segments the target image to generate first text information, a first accuracy indicator, and coordinates for each character included in the target image. The first optical character recognition device 200 transmits the first generated information to the electronic data generation device 100.

[0087] In step S102, the electronic data generation device 100 stores the first generation information for each character in the target image information D101a in association with the target image.

[0088] In step S103, the electronic data generation device 100 determines whether the first reference indicator of the character included in the target image is lower than the first reference.

[0089] If it is determined that the first reference indicator is equal to or greater than the first reference (step S103: NO), in step S104, the electronic data generation device 100 identifies the characters indicated by the first text information as characters included in the target image.

[0090] If it is determined that the first reference indicator is lower than the first reference (step S103: YES), in step S105, the electronic data generation device 100 refers to the target image information D101a and identifies line information including first text information corresponding to the first reference indicator determined to be lower than the first reference. Specifically, the electronic data generation device 100 identifies the first text information and the first validity indicator shown in FIG. 6(a).

[0091] In step S106, the electronic data generation device 100 identifies a line image (specific range image) corresponding to the line information from the target image based on the coordinates included in the identified line information. The electronic data generation device 100 transmits the identified line image to the second optical character recognition device 300.

[0092] In step S107, the second optical character recognition device 300 segments the line image to generate second text information, a second accuracy indicator, and coordinates for each character included in the line image. The second optical character recognition device 300 transmits the second generated information to the electronic data generation device 100.

[0093] In step S108, the electronic data generation device 100 stores the second generation information for each character in the specific range image information D101b in association with the specific range image.

[0094] In step S109, the electronic data generation device 100 determines whether the second reference indices in the specific range image are larger than the second reference. Specifically, it determines whether all of the second validity indices (second reference indices) of the line information shown in FIG. 6(b) exceed a threshold value (second reference).

[0095] If it is determined that the second reference indicator is equal to or greater than the second reference (step S109: YES), in step S110, the electronic data generation device 100 identifies the character indicated by the second text information included in the line information as the character included in the line image.

[0096] If it is determined that the second reference index is lower than the second reference index (step S109: NO), in step S111, the electronic data generation device 100 compares the first accuracy index for the line information corresponding to the specific range image with the second accuracy index. Specifically, the electronic data generation device 100 compares the smallest value of the first accuracy indexes shown in Figure 6(a) ("0.32" in Figure 6(a)) with the smallest value of the second accuracy indexes shown in Figure 6(b) ("0.57" in Figure 6(b)).

[0097] In step S112, if the electronic data generation device 100 determines that the smallest value of the first accuracy indexes is greater than the smallest value of the second accuracy indexes, it identifies the line information including the first text information (line information in FIG. 6(a)) as a character included in the line image. On the other hand, if the electronic data generation device 100 determines that the smallest value of the second accuracy indexes is greater than the smallest value of the first accuracy indexes, it identifies the line information including the second text information (line information in FIG. 6(b)) as a character included in the line image.

[0098] This makes it possible to adopt a character string that is estimated to have been recognized more accurately from a predetermined range of character strings that includes characters with low accuracy in the first character recognition by the first optical character recognition device 200 and a character string in the second character recognition by the second optical character recognition device 300.

[0099] In step S112, the electronic data generation device 100 may compare the average value of the first accuracy index ("average value" in FIG. 6(a)) with the average value of the second accuracy index ("average value" in FIG. 6(b)). In this case, the electronic data generation device 100 identifies the line information (line information in FIG. 6(b)) corresponding to the larger average value as a character included in the line image.

[0100] In step S113, the electronic data generation device 100 displays on the display unit of the user terminal 400 a screen T10 shown in FIG. 4 that allows a comparison between the target image and the result of character recognition of the target image.

[0101] If multiple pieces of line information are specified in step S106, the electronic data generation system 10 repeatedly executes steps S106 to S112 the number of times equal to the number of pieces of line information specified.

[0102] In this way, in the electronic data generation system 10, it is desirable to perform character recognition by the second optical character recognition device 300, which is capable of more appropriately performing character recognition on a line-by-line basis, for lines that are determined to have low character recognition accuracy as a result of character recognition by the first optical character recognition device 200, which is capable of more appropriately performing character recognition on a page-by-page basis.The electronic data generation system 10 then compares the character recognition results by the first optical character recognition device 200 with the character recognition results by the second optical character recognition device 300, and adopts the character recognition result with the higher accuracy.In other words, the electronic data generation device 100 performs character recognition on different character recognition ranges using two different optical character recognition devices, thereby making it possible to achieve highly accurate character recognition for images with low character recognition accuracy.

[0103] ===Hardware Configuration=== An example of a hardware configuration for implementing the electronic data generation device 100, the first optical character recognition device 200, the second optical character recognition device 300, and the user terminal 400 on a computer will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of the hardware configuration of a computer.

[0104] As shown in FIG. 7, the computer 1000 includes a processor 1001, a memory 1002, a storage device 1003, an input I / F unit 1004, a data I / F unit 1005, a communication I / F unit 1006, and a display unit 1007.

[0105] The processor 1001 is a control unit that controls various processes in the computer 1000 by executing programs stored in the memory 1002 .

[0106] The memory 1002 is a storage medium such as a RAM (Random Access Memory), etc. The memory 1002 temporarily stores the program code of the program executed by the processor 1001 and data required when the program is executed.

[0107] The storage device 1003 is a non-volatile storage medium such as a hard disk drive (HDD), flash memory, etc. The storage device 1003 stores an operating system and various programs for realizing the above-mentioned components.

[0108] The input I / F unit 1004 is a device for receiving input from a user. Specific examples of the input I / F unit 1004 include a keyboard, a mouse, a touch panel, various sensors, and a wearable device. The input I / F unit 1004 may be connected to the computer 1000 via an interface such as a USB (Universal Serial Bus).

[0109] The data I / F unit 1005 is a device for inputting data from outside the computer 1000. A specific example of the data I / F unit 1005 is a drive device for reading data stored in various storage media. The data I / F unit 1005 may be provided outside the computer 1000. In this case, the data I / F unit 1005 is connected to the computer 1000 via an interface such as a USB.

[0110] The communication I / F unit 1006 is a device for performing data communication via the Internet N, either wired or wirelessly, with devices external to the computer 1000. The communication I / F unit 1006 may be provided external to the computer 1000. In this case, the communication I / F unit 1006 is connected to the computer 1000 via an interface such as a USB.

[0111] The display unit 1007 is a device for displaying various types of information. Specific examples of the display unit 1007 include a liquid crystal display, an organic EL (Electro-Luminescence) display, and a display of a wearable device. The display unit 1007 may be provided outside the computer 1000. In this case, the display unit 1007 is connected to the computer 1000 via, for example, a display cable. Furthermore, when a touch panel is used as the input I / F unit 1004, the display unit 1007 can be configured as an integrated unit with the input I / F unit 1004.

[0112] ===Summary=== <1> The electronic data generation system 10 of this embodiment includes a target image acquisition unit 102 that acquires a target image including characters from a predetermined device, a first information acquisition unit 103 that acquires, from a first optical character recognition device 200 that recognizes each of a plurality of characters included in the target image as first text information, first text information for each of the plurality of characters and a first accuracy index indicating the degree of accuracy of recognition for the character indicated by the first text information, and a first reference index based on at least one of the first accuracy indexes for the plurality of characters that is higher than a first reference regarding the accuracy of recognition of the characters included in the image. When the first reference index is determined to be lower than the first standard, the electronic data generation system 10 further includes a second information acquisition unit 107 that acquires second text information about characters included in the specific range image from a second optical character recognition device 300 different from the first optical character recognition device 200, the second information acquisition unit 107 recognizing each of at least one character included in the specific range image as second text information when the first reference index is determined to be lower than the first standard, and a character identification unit 109 that identifies characters included in the target image based on the first text information and the second text information. This allows the electronic data generation system 10 to achieve highly accurate character recognition by performing character recognition on an image with low accuracy of character recognition using multiple optical character recognition devices with different target ranges for character recognition (for example, the first optical character recognition device 200 performs character recognition on a page basis and the second optical character recognition device 300 performs character recognition on a line basis).

[0113] <2> Furthermore, the second information acquisition unit 107 in the electronic data generation system 10 of this embodiment acquires, from the second optical character recognition device 300, second text information for each of at least one character included in the specific range image and a second accuracy index indicating the degree of recognition accuracy for the character indicated by the second text information, and the character identification unit 109 identifies the character included in the specific range image based on the result of determining the magnitude relationship between a second reference index based on at least one of the second accuracy indexes for each of the at least one character included in the specific range image and a second reference regarding the recognition accuracy of the character included in the image. This enables the electronic data generation system 10 to appropriately recognize a character string in a predetermined range, including characters that were recognized with low accuracy in the first character recognition by the first optical character recognition device 200, in the second character recognition by the second optical character recognition device 300.

[0114] <3> Furthermore, when it is determined that the second reference index for all characters included in the specific range image is equal to or greater than the second reference, the character identification unit 109 of the electronic data generation system 10 in this embodiment identifies characters indicated by the second text information corresponding to the second accuracy index as characters included in the specific range image. This allows the electronic data generation system 10 to properly recognize character strings in the second character recognition by the second optical character recognition device 300 for character strings in a predetermined range that include characters with low accuracy in the first character recognition by the first optical character recognition device 200.

[0115] <4> Furthermore, when the character identification unit 109 of the electronic data generation system 10 in this embodiment determines that the second reference index for at least one character among the characters included in the specific range image is lower than the second reference, it identifies the characters included in the specific range image based on the result of determining the magnitude relationship between the first accuracy index for at least one character included in the range corresponding to the specific range image in the target image and the second accuracy index for at least one character included in the specific range image. This allows the electronic data generation system 10 to achieve more appropriate character recognition using a character string that is estimated to have been recognized more accurately between a character string in a predetermined range that includes characters with low accuracy in the first character recognition by the first optical character recognition device 200 and a character string in the second character recognition by the second optical character recognition device 300.

[0116] <5> Furthermore, the target image acquisition unit 102 of the electronic data generation system 10 in this embodiment acquires a target image, which is an image in units of one page, and the information transmission unit 106 transmits a specific range image, which is an image in units of lines of text included in the target image, to the second optical character recognition device 300. As a result, in the electronic data generation system 10, for example, for a line image for which the first optical character recognition device 200 has low character recognition accuracy, by using the second optical character recognition device 300, which has high character recognition accuracy for line images, it is possible to improve the accuracy of character recognition.

[0117] <6> Furthermore, the character identification unit 109 of the electronic data generation system 10 in this embodiment identifies characters indicated by the first text information corresponding to the first accuracy index as characters included in the specific range image when inputting the specific range image to the second optical character recognition device 300 satisfies the conditions for inputting the specific range image to the second optical character recognition device 300. As a result, when the cost of character recognition processing by the second optical character recognition device 300 is higher than the cost of character recognition processing by the first optical character recognition device 200, for example, or exceeds a certain cost, the electronic data generation system 10 can achieve cost reduction by performing character recognition using a less expensive optical character recognition device.

[0118] <7> Moreover, the electronic data generation system 10 in this embodiment further includes a display processing unit 110 that displays the target image in a first display area T11 of the screen T10, and displays second text information indicating characters included in the specific range image in a second display area T12 different from the first display area T11 on the screen T10. This allows the electronic data generation system 10 to provide the user with a correspondence between the target image and the first text information and second text information, which are the results of character recognition of a predetermined range of the target image, and enables the user to easily understand misrecognition of the target image, etc.

[0119] <8> Furthermore, the display processing unit 110 in the electronic data generation system 10 of this embodiment displays, in the second display area T12, second text information indicating characters included in the specific range image and first text information indicating characters included in the target image excluding the characters included in the specific range image, and displays the second text information in an identifiable manner. This allows the electronic data generation system 10 to display the second text information in the second optical character recognition device 300 in a manner that allows the user to easily identify it, thereby enabling the user to easily grasp the degree of recognition error in the target image, etc.

[0120] <9> Furthermore, the display processing unit 110 of the electronic data generation system 10 in this embodiment displays in a predetermined display area on the screen the number of inputs, which is the number of times the specific range image has been input to the second optical character recognition device 300. This allows the electronic data generation system 10 to allow the user to easily grasp the number of times character recognition has been performed by the second optical character recognition device 300. [Explanation of symbols]

[0121] 10...electronic data generation system, 100...electronic data generation device, 101...memory unit, 102...target image acquisition unit, 103...first information acquisition unit, 104...first judgment unit, 105...specific range identification unit, 106...information transmission unit, 107...second information acquisition unit, 108...second judgment unit, 109...character identification unit, 110...display processing unit, 200...first optical character recognition device, 300...second optical character recognition device, 400...user terminal.

Claims

1. a target image acquisition unit that acquires a target image including characters from a predetermined device; a first information acquisition unit that acquires, from a first optical character recognition device that recognizes each of a plurality of characters included in the target image as first text information, the first text information for each of the plurality of characters and a first accuracy index that indicates a degree of accuracy of recognition of the character indicated by the first text information; a second information acquisition unit that acquires, when a first reference indicator based on at least one of the first accuracy indicators for the plurality of characters is determined to be lower than a first standard related to the accuracy of recognition of characters included in an image, the second information acquisition unit acquires the second text information about characters included in the specific range image from a second optical character recognition device different from the first optical character recognition device, the second information acquisition unit being a specific range image that is an image of a partial range of the target image and that includes characters corresponding to the first reference indicator determined to be lower than the first standard, and that recognizes each of at least one character included in the specific range image as second text information; a character identification unit that identifies characters included in the target image based on the first text information and the second text information; An information processing system comprising:

2. the second information acquisition unit acquires, from the second optical character recognition device, the second text information for each of at least one character included in the specific range image, and a second accuracy index indicating a degree of recognition accuracy for the character indicated by the second text information; the character identification unit identifies a character included in the specific range image based on a result of determining a magnitude relationship between a second reference index based on at least one of the second accuracy indexes for each of at least one character included in the specific range image and a second reference index related to accuracy of recognition of the character included in the image; The information processing system according to claim 1 .

3. when it is determined that the second reference indicators for all characters included in the specific range image are equal to or greater than the second reference, the character identification unit identifies characters indicated by the second text information corresponding to the second accuracy indicators as characters included in the specific range image; The information processing system according to claim 2 .

4. When it is determined that the second reference index for at least one character among the characters included in the specific range image is lower than the second reference, the character identification unit identifies the character included in the specific range image based on a result of determining a magnitude relationship between the first accuracy index of at least one character included in a range corresponding to the specific range image in the target image and the second accuracy index of at least one character included in the specific range image. The information processing system according to claim 2 .

5. the target image acquisition unit acquires the target image, which is an image in units of one page; the second information acquisition unit acquires, from the second optical character recognition device, the second text information about characters included in the specific range image, which is an image of sentences in units of lines included in the target image; The information processing system according to claim 1 .

6. the character identification unit, when inputting the specific range image to the second optical character recognition device satisfies a condition related to inputting the specific range image to the second optical character recognition device, identifies a character indicated by the first text information corresponding to the first accuracy index as a character included in the specific range image; The information processing system according to claim 1 .

7. Displaying the target image in a first display area of ​​a screen; a display processing unit that displays the second text information indicating characters included in the specific range image in a second display area different from the first display area on the screen; The information processing system according to claim 1 further comprising:

8. The display processing unit displaying, in the second display area, the second text information indicating characters included in the specific range image and the first text information indicating characters included in the target image excluding the characters included in the specific range image; identifiably displaying the second text information; The information processing system according to claim 7 .

9. the display processing unit displays, in a predetermined display area on the screen, an input count, which is the number of times the specific range image has been input to the second optical character recognition device; The information processing system according to claim 7 .

10. The computer Acquiring a target image including text from a predetermined device; acquiring, from a first optical character recognition device that recognizes each of a plurality of characters included in the target image as first text information, the first text information for each of the plurality of characters and a first accuracy index that indicates a degree of accuracy of recognition of the character indicated by the first text information; When it is determined that a first reference indicator based on at least one of the first accuracy indicators for the plurality of characters is lower than a first standard related to the accuracy of recognition of characters included in an image, a specific range image is an image of a partial range of the target image, including characters corresponding to the first reference indicator determined to be lower than the first standard, and the second text information for characters included in the specific range image is acquired from a second optical character recognition device different from the first optical character recognition device, the second optical character recognition device recognizing each of at least one character included in the specific range image as second text information; Identifying characters included in the target image based on the first text information and the second text information; An information processing method that performs the above.

11. On the computer, Acquiring a target image including text from a predetermined device; acquiring, from a first optical character recognition device that recognizes each of a plurality of characters included in the target image as first text information, the first text information for each of the plurality of characters and a first accuracy index that indicates a degree of accuracy of recognition of the character indicated by the first text information; When it is determined that a first reference indicator based on at least one of the first accuracy indicators for the plurality of characters is lower than a first standard related to the accuracy of recognition of characters included in an image, a specific range image is an image of a partial range of the target image, including characters corresponding to the first reference indicator determined to be lower than the first standard, and the second text information for characters included in the specific range image is acquired from a second optical character recognition device different from the first optical character recognition device, the second optical character recognition device recognizing each of at least one character included in the specific range image as second text information; Identifying characters included in the target image based on the first text information and the second text information; A program that executes the following.

Images