Information processing system, information processing method, and program

Abstract

To provide a system capable of appropriately extracting information from an image. [Solution] In an information processing system, an electronic data generation device includes a second information acquisition unit that acquires second text information from a second optical character recognition device different from the first optical character recognition device that recognizes at least one character included in a specific range image, which is an image of a portion of the target image, as second text information, including a character corresponding to the first standard indicator determined to be lower than the first standard, when a first accuracy index indicating the degree of recognition accuracy for characters indicated by the first text information is determined to be lower than a first standard for the accuracy of recognition of characters included in an image from a first optical character recognition device that recognizes each of a plurality of characters included in a target image as first text information, and a character identification unit that identifies a character included in the target image based on the first text information and the second text information.

Description

[Technical field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] A system for extracting information from document images using a machine learning module is disclosed (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-79439 A Summary of the Invention [Problem to be solved by the invention]

[0004] The document data extraction system described in Patent Document 1 obtains image data associated with a document and extracts metadata from the image data by optical character recognition. The metadata specifies a string of text content items and text content item features associated with each text content item in the string of text content items. The document data extraction system uses a machine learning module to determine one or more text content items associated with a key based on the string of text content items and the text content item features. This enables the document data extraction system to extract information from the document image.

[0005] However, if metadata cannot be properly generated from image data using a single optical character recognition device that performs optical character recognition, a problem occurs in that information cannot be properly extracted from document images.

[0006] Therefore, in order to solve the above problems, an object of the present invention is to provide a system capable of appropriately extracting information from an image. [Means for solving the problem]

[0007] An information processing system according to one aspect of the present invention includes a target image acquisition unit that acquires a target image including characters from a predetermined device, a first information acquisition unit that acquires, from a first optical character recognition device that recognizes each of a plurality of characters included in the target image as first text information, the first text information for each of the plurality of characters and a first accuracy index indicating a degree of recognition accuracy for the characters indicated by the first text information, and a first reference index based on at least one of the first accuracy indexes for the plurality of characters that is higher than a first reference regarding the accuracy of recognition of characters included in an image. When the first reference indicator is determined to be low, a specific range image is an image of a partial range of the target image including characters corresponding to the first reference indicator determined to be lower than the first standard, and at least one character included in the specific range image is recognized as second text information. A second information acquisition unit acquires the second text information about characters included in the specific range image from a second optical character recognition device different from the first optical character recognition device, and a character identification unit identifies characters included in the target image based on the first text information and the second text information.

[0008] An information processing method according to one embodiment of the present invention includes a computer executing the steps of: acquiring a target image including characters from a specified device; acquiring, from a first optical character recognition device that recognizes each of a plurality of characters included in the target image as first text information, first text information for each of the plurality of characters and a first accuracy index indicating a degree of recognition accuracy for the character indicated by the first text information; when a first reference index based on at least one of the first accuracy indexes for the plurality of characters is determined to be lower than a first reference standard for accuracy of recognition of characters included in an image, acquiring second text information for characters included in the specific range image from a second optical character recognition device different from the first optical character recognition device that recognizes each of at least one character included in the specific range image as second text information; and identifying characters included in the target image based on the first text information and the second text information.

[0009] A program according to one embodiment of the present invention causes a computer to execute the following steps: acquire a target image including characters from a specified device; acquire, from a first optical character recognition device that recognizes each of a plurality of characters included in the target image as first text information, first text information for each of the plurality of characters and a first accuracy index indicating a degree of recognition accuracy for the characters indicated by the first text information; when a first reference index based on at least one of the first accuracy indexes for the plurality of characters is determined to be lower than a first reference standard for accuracy of recognition of characters included in an image, acquire second text information for characters included in the specific range image from a second optical character recognition device different from the first optical character recognition device that recognizes each of at least one character included in the specific range image as second text information; and identify characters included in the target image based on the first text information and the second text information. Effect of the Invention

[0010] According to the present invention, it is possible to provide a system capable of appropriately extracting information from an image. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing an overview of an electronic data generation system. [Diagram 2] 1 is a database showing an example of target image information. [Diagram 3] 1 is a database showing an example of specific range image information. [Figure 4] FIG. 4 is a diagram showing an example of a screen displayed on a display unit. [Diagram 5] 1 is a flowchart showing a processing procedure of the electronic data generation system. [Figure 6]11 is a table showing validity indicators for line information that is text information of one line of a character string. [Figure 7] FIG. 2 illustrates an example of a hardware configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] An electronic data generation system 10 according to an embodiment of the present invention will be described in detail below with reference to the drawings. However, the embodiment described below is merely an example, and is not intended to exclude the application of various modifications and techniques not specifically described below. In other words, the present invention can be implemented by modifying it in various ways or combining the various embodiments without departing from the spirit of the invention. In addition, in the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

[0013] In this embodiment, the terms "unit", "device", and "system" do not simply mean physical means, but also include cases where the functions of the "unit", "device", and "system" are realized by software. The functions of one "unit", "device", or "system" may be realized by two or more physical means or devices, and the functions of two or more "units", "devices", or "systems" may be realized by one physical means or device. Furthermore, the various functions described below of each of the multiple devices constituting the electronic data generation system 10 may be configured to be executed by other devices in the multiple devices.

[0014] ===Outline of Electronic Data Generation System 10=== <<Configuration Overview>> An overview of the electronic data generation system 10 will be described with reference to Fig. 1. Fig. 1 is a diagram showing an overview of the electronic data generation system 10.

[0015] The electronic data generation system 10 is a system that uses multiple optical character recognition devices to accurately generate digital documents from images. Specifically, the electronic data generation system 10 generates text information, which is digital data, from images specified in graphic formats such as JPEG (Joint Photographic Experts Group), TIFF (Tagged Image File Format), and PNG (Portable Network Graphics) and images in PDF (Portable Document Format) data (hereinafter referred to as "target images") through multiple optical character recognition devices.

[0016] The target image is, for example, an image of various contracts, papers, etc. For convenience, the following description will be given assuming that the target image is, as an example, an image of a single page of a contract.

[0017] The electronic data generation system 10 includes, for example, an electronic data generation device 100, a first optical character recognition device 200, a second optical character recognition device 300, and a user terminal 400.

[0018] The electronic data generating device 100 is a device that outputs a digital document that is the result of character recognition of a target image, based on the results of character recognition of the target image by each of two different optical character recognition devices.

[0019] The first optical character recognition device 200 is a device that performs character recognition on a target image.

[0020] The second optical character recognition device 300 is a device that executes character recognition on an image within a predetermined range of a target image acquired from the electronic data generation device 100.

[0021] The electronic data generating device 100, the first optical character recognition device 200, and the second optical character recognition device 300 may be, for example, a cloud computer, a server computer, a personal computer (e.g., a desktop, a laptop, a tablet, etc.), a media computer platform (e.g., a cable, a satellite set-top box, a digital video recorder), a handheld computer device (e.g., a PDA, an email client, etc.), or other types of computers or communication platforms. At least a part of the processing in the electronic data generating device 100, the first optical character recognition device 200, and the second optical character recognition device 300 may be realized by one or more computers (for example, but not limited to, cloud computing configured by one or more computers).

[0022] The user terminal 400 is a device that receives operational inputs from a user and displays various types of information.

[0023] The user terminal 400 may be, for example, a smartphone, a mobile phone (feature phone), a personal computer (e.g., desktop, laptop, tablet, etc.), a media computing platform (e.g., cable, satellite set-top box, digital video recorder), a handheld computing device (e.g., PDA (Personal Digital Assistant), email client, etc.), a wearable device (glasses-type device, watch-type device, etc.), or another type of computer or communication platform.

[0024] <<Processing Overview>> An overview of the processing of the electronic data generation system 10 will be described with reference to FIG.

[0025] First, in step S10, the electronic data generation device 100 transmits a target image acquired from a predetermined device to the first optical character recognition device 200.

[0026] In step S11, the first optical character recognition device 200 recognizes characters included in a target image (e.g., an image of one page) and generates text information about the recognized characters (hereinafter referred to as "first text information"). The first text information may include the coordinates of the characters on the target image. At this time, the first optical character recognition device 200 generates a degree of recognition accuracy for the characters indicated by the generated first text information (hereinafter referred to as "first accuracy index"). For convenience, the first text information, the first accuracy index, and the coordinates may be collectively referred to as "first generated information" below.

[0027] The first optical character recognition device 200 transmits the first generated information to the electronic data generation device 100 .

[0028] In step S12, if the electronic data generation device 100 determines that a reference index (hereinafter referred to as the "first reference index") based on at least one of the first accuracy indexes for multiple characters is lower than a standard (hereinafter referred to as the "first standard") regarding the accuracy of recognizing characters contained in an image, it identifies an image of a predetermined range of the target image (hereinafter referred to as a "specific range image") that contains characters corresponding to the reference index.

[0029] The first reference index may be, for example, a first accuracy index for each of a plurality of characters, or an average value of the first accuracy indexes for a plurality of characters contained in a line-unit image or a block-unit image.

[0030] The first criterion may be, for example, a threshold value comparable to the first accuracy measure.

[0031] The specific range image may be, for example, an image of a single sentence (hereinafter referred to as a "block image") obtained by segmenting a portion of a page-unit target image, an image of one line (hereinafter referred to as a "line image"), or a character image that is an image of one character. For convenience, the specific range image will be described below as a "line image."

[0032] The electronic data generating device 100 transmits the specific range image to the second optical character recognition device 300. That is, the electronic data generating device 100 causes an optical character recognition device different from the optical character recognition device that performed character recognition on the target image to perform character recognition again on a specific range image (e.g., a line image) of, for example, a part of the target image, including characters determined to have low accuracy of character recognition.

[0033] In step S13, the second optical character recognition device 300 recognizes characters included in the specific range image and generates text information about the recognized characters (hereinafter referred to as "second text information"). The second text information may include the coordinates of the characters on the target image. At this time, the second optical character recognition device 300 generates a degree of recognition accuracy for the characters indicated by the generated second text information (hereinafter referred to as "second accuracy index"). Note that, hereinafter, the second text information, the second accuracy index, and the coordinates may be collectively referred to as "second generated information".

[0034] The second optical character recognition device 300 transmits the second generated information to the electronic data generation device 100 .

[0035] In step S14, the electronic data generation device 100 identifies characters contained in the specific range image based on a comparison result (hereinafter referred to as the "first comparison result") between a reference indicator (hereinafter referred to as the "second reference indicator") based on at least one of the second accuracy indicators for each of at least one character contained in the specific range image and a standard regarding the accuracy of recognition of characters contained in the image (hereinafter referred to as the "second standard").

[0036] The second reference index may be, for example, a second accuracy index for each character included in the specific range image, or may be an average value of the second accuracy indexes for multiple characters included in the specific range image.

[0037] The second criterion may be, for example, a threshold value comparable to the second accuracy measure.

[0038] The electronic data generation device 100 transmits to the user terminal 400 a screen including information indicating characters included in the identified specific range image (hereinafter, referred to as a "character recognition result").

[0039] As described above, when the accuracy of character recognition by the first optical character recognition device 200 for characters included in a target image on a page basis is low, the electronic data generation device 100 obtains the results of character recognition of a specific range image (here, as an example, a line image on a line basis) of a part of the target image including the characters using the second optical character recognition device 300, which is different from the first optical character recognition device 200, and identifies the characters included in the target image based on the results.

[0040] In other words, the electronic data generation device 100 uses multiple optical character recognition devices with different target ranges for character recognition (for example, the first optical character recognition device 200 performs character recognition on a page-by-page basis, and the second optical character recognition device 300 performs character recognition on a line-by-line basis) to perform character recognition on images with low character recognition accuracy, thereby making it possible to achieve highly accurate character recognition.

[0041] Furthermore, in the electronic data generation system 10, instead of performing all character recognition in the target range using the second optical character recognition device 300, which has a higher cost of performing character recognition than the first optical character recognition device 200, the accuracy of character recognition is lower with the first optical character recognition device 200, and character recognition is performed in a narrower range than the target range, thereby making it possible to increase the accuracy of character recognition and reduce the cost of character recognition.

[0042] The specific range image transmitted to the second optical character recognition device 300 is not limited to being a line image, but may be an image of one page, a block image, or a character image.

[0043] Furthermore, the electronic data generation system 10 may further include at least one optical character recognition device in addition to the first optical character recognition device 200 and the second optical character recognition device 300. In this case, the optical character recognition device may be one that performs character recognition on an image in the same range as either the first optical character recognition device 200 or the second optical character recognition device 300, or may be one that performs character recognition on an image in a narrower range than the range in which the first optical character recognition device 200 and the second optical character recognition device 300 perform character recognition. This allows the electronic data generation system 10 to achieve more accurate character recognition.

[0044] ===Electronic Data Generation Device 100=== As shown in FIG. 1, the electronic data generation device 100 includes a memory unit 101, a target image acquisition unit 102, a first information acquisition unit 103, a first judgment unit 104, a specific range identification unit 105, an information transmission unit 106, a second information acquisition unit 107, a second judgment unit 108, a character identification unit 109, and a display processing unit 110.

[0045] The storage unit 101 includes, for example, target image information D101a and specific range image information D101b.

[0046] The target image information D101a will be described with reference to Fig. 2. Fig. 2 is a database showing an example of the target image information 101a. The target image information 101a is a database in which data relating to a target image is stored.

[0047] As shown in FIG. 2, the target image information D101a includes items such as [target image ID], [target image], [block information], [line information], [character information], [coordinates], and [first accuracy index]. [Target image ID] stores identification information that can uniquely identify the target image. [Target image] stores the target image. [Block information] stores text information of a block image that indicates a group of sentences in the target image (hereinafter referred to as "block information"). [Line information] stores text information for each line included in the block image (hereinafter referred to as "line information"). [Character information] stores text information of characters included in a line (hereinafter referred to as "character information"). [Coordinates] stores coordinates in the target image of characters (character information) indicated by each piece of first text information. [First accuracy index] stores a first accuracy index indicating the degree of accuracy of characters indicated by each piece of first text information.

[0048] The specific range image information D101b will be described with reference to Fig. 3. Fig. 3 is a database showing an example of the specific range image information 101b. The specific range image information 101b is a database in which data relating to specific range images is stored.

[0049] As shown in FIG. 3, the specific range image information D101b includes items such as [specific range image ID], [specific range image], [second text information], [coordinates], and [second accuracy index]. [Specific range image ID] stores identification information that can uniquely identify the specific range image. [Specific range image] stores the specific range image. [Second text information] stores second text information of the character image included in the specific range image (line image in FIG. 3). [Coordinates] stores the coordinates in the target image or the coordinates in the specific range image of the characters indicated by the second text information. [Second accuracy index] stores a second accuracy index that indicates the degree of accuracy of the characters indicated by the second text information.

[0050] The target image acquisition unit 102 acquires a target image from a predetermined device. The predetermined device is, for example, a scanner device capable of acquiring printout on paper as an image, or a server device that stores the target image. The target image acquisition unit 102 may transmit the acquired target image to the first optical character recognition device 200.

[0051] Returning to Fig. 1, the first information acquisition unit 103 acquires first generated information about the target image from the first optical character recognition device 200. Specifically, the electronic data generation device 100 acquires, for example, first text information (including coordinates) and a first accuracy index about each character image included in the target image from the first optical character recognition device 200. The first generated information is stored in the target image information D101a.

[0052] The first determination unit 104 determines whether the first reference index is lower than the first reference. Specifically, when the threshold value as the first reference is "0.6" and the image in a predetermined range of the target image (for example, a line image) is "ABC", if the first accuracy indexes of the first text information "A", "B", and "D" (character image "C" is recognized as "D") are "0.99", "0.99", and "0.55", respectively, the first determination unit 104 determines that the first reference index (here, the first accuracy index "0.55" corresponding to "D") in the image in the predetermined range is lower than the first reference (here, the threshold value "0.6").

[0053] In other words, if at least one (or the average value) of the first accuracy indices corresponding to the first text information in a specified range of images (e.g., block images, line images, or character images) included in the target image is lower than a threshold value, the electronic data generation device 100 may determine that accurate character recognition has not been performed for the specified range of images.

[0054] The specific range identifying unit 105 identifies a specific range image, which is an image of a partial range of the target image, including characters corresponding to the first reference indicator that is determined to be lower than the first standard based on the determination result (hereinafter referred to as the "first determination result") by the first determination unit 104. In this case, the specific range identifying unit 105 identifies the coordinates of each character image included in the target image acquired from the first optical character recognition device 200, and identifies a specific range image (e.g., a line image) including characters corresponding to the first reference indicator based on the coordinates.

[0055] Here, an overview of an example of a process for identifying a line image (hereinafter, referred to as "line identification process") will be described. In the line identification process, first, a black dot image at the left end of a target image is identified. Next, in the line identification process, a histogram (for example, the horizontal axis is the distance from the left end of the target image and the vertical axis is the number of dot images) is generated while identifying black dots in a predetermined width in the height direction from the identified dot image in the horizontal direction to the right. Next, a histogram is generated while similarly identifying black dot images in a predetermined width by shifting the angle with respect to the horizontal. Then, in the line identification process, the coordinates of the upper left dot and the lower right dot are identified based on the histogram, thereby identifying the coordinates of the range of the line image. This makes it possible to appropriately identify a predetermined group of images including multiple characters.

[0056] In addition, in the histogram, when the first distance range in which the black dot image is identified and the second distance range in which the black dot image is identified exceed a predetermined distance, the range in which the black dot image in the first distance range is identified as a first row image, and the range in which the black dot image in the second distance range is identified as a second row image different from the first row image. This makes it possible to identify row images included in different block images, even though they are in the same row, as different rows.

[0057] The information transmitting unit 106 transmits the specified specific range image (eg, a line image) to the second optical character recognition device 300.

[0058] The second information acquisition unit 107 acquires second generation information about characters included in the specific range image from the second optical character recognition device 300. Specifically, the electronic data generation device 100 acquires, for example, second text information (including coordinates) and second accuracy indexes (for example, "A: 0.99", "B: 0.98", "C: 0.99") about each of the character images (for example, "A", "B", "C") included in a line image (for example, "ABC") that is the specific range image from the second optical character recognition device 200.

[0059] The second determination unit 108 determines whether the second reference index is larger than the second reference. Specifically, the second determination unit 108 determines whether all the second accuracy indexes for the characters included in the specific range image are equal to or larger than a predetermined threshold. For example, when the predetermined threshold is "0.6" and the specific range image (e.g., a line image) is "ABC", if the second accuracy indexes for the second text information "A", "B", and "C" are "0.99", "0.99", and "0.98", respectively, the second determination unit 108 determines that the second reference index is equal to or larger than the second reference. The second determination unit 108 may determine whether the average value of the second accuracy indexes for the characters included in the specific range image is equal to or larger than a predetermined threshold.

[0060] The second determination unit 108 may determine that the second reference index is lower than the second reference when any one of the second accuracy indices for the specific range image is lower than a predetermined threshold. For example, when the specific range image (e.g., a line image) is "ABC", if the second accuracy indices of the second text information "A", "B", and "C" are "0.99", "0.99", and "0.50", respectively, the second determination unit 108 determines that the second reference index is lower than the second reference.

[0061] In this way, the electronic data generation device 100 determines that the second text information is able to recognize characters more accurately for the character image in the target image than the first text information, for example, when all of the second accuracy indicators for the specific range image exceed a predetermined threshold.

[0062] Character identification unit 109 identifies characters included in the specific range image (i.e., target image) based on the determination result (hereinafter referred to as the "second determination result") by second determination unit 108. Specifically, when the second reference index is equal to or greater than a second reference, character identification unit 109 identifies characters indicated by the second text information corresponding to the second accuracy index as characters included in the specific range image.

[0063] For example, if the second accuracy index (second reference index) of the second text information "A", "B", and "C" are "0.99", "0.99", and "0.98", respectively, the character identification unit 109 identifies "ABC" indicated by the second text information as a character string in the specific range image because the second reference index is greater than or equal to the second reference (e.g., a threshold value of "0.6").

[0064] This makes it possible for a character string within a certain range, including characters that are recognized with low accuracy in the first character recognition by the first optical character recognition device 200, to be properly recognized in the second character recognition by the second optical character recognition device 300.

[0065] On the other hand, when the second reference index is lower than the second reference index, the character identification unit 109 identifies, based on the result of determining the magnitude relationship between the first accuracy index and the second accuracy index, either the character indicated by the first text information corresponding to the first accuracy index or the character indicated by the second text information corresponding to the second accuracy index as a character in the specific range image.

[0066] Specifically, the character identification unit 109 identifies the minimum value indicating the higher value among the minimum value of the second accuracy index of each character image included in the specific range image acquired from the second optical character recognition device 300 and the minimum value of the first accuracy index corresponding to the first text information of the image corresponding to the specific range image acquired from the first optical character recognition device 200. The character identification unit 109 identifies the character indicated by the text information (the first text information or the second text information) corresponding to the accuracy index indicating the identified minimum value as the character included in the specific range image.

[0067] For example, for the line image "ABC", if the first accuracy indexes of the first text information "A", "F", and "D" (here, character image "B" is recognized as "F" and "C" is recognized as "D") are "0.99", "0.40", and "0.55", and the second accuracy indexes of the second text information "A", "B", and "E" (character image "C" is recognized as "E") are "0.99", "0.99", and "0.50", respectively, the character identification unit 109 identifies that the minimum value "0.50" of the second accuracy index is higher than the minimum value "0.40" of the first accuracy index. In this case, the character identification unit 109 identifies "ABE" indicated by the second text information as a character string in the specific range image.

[0068] This makes it possible to adopt a character string that is estimated to have been recognized more accurately between a predetermined range of character strings including characters with low accuracy in the first character recognition by the first optical character recognition device 200 and a character string in the second character recognition by the second optical character recognition device 300.

[0069] In the above description, the minimum value indicating the higher value of the minimum value of the second accuracy index and the minimum value of the first accuracy index is specified, but the present invention is not limited to this. For example, the character specification unit 109 may use an average value instead of the minimum value, and in this case, the character indicated by the text information corresponding to the accuracy index (second reference index) indicating the higher average value may be specified as the character of the specific range image. For example, the electronic data generation device 100 can reduce the user's effort in correction by executing a process of specifying a minimum value indicating a higher value of the minimum values, while by executing a process of specifying a higher average value of the average values, the electronic data generation device 100 can reduce the user's effort in correction since it is possible to adopt text that is only required to be well organized overall.

[0070] As described above, when the accuracy of character recognition by the first optical character recognition device 200 for an image in a predetermined range (for example, in units of lines) of the target image is low, the electronic data generation device 100 obtains the result of character recognition of the image in the predetermined range by the second optical character recognition device 300 different from the first optical character recognition device 200. That is, the electronic data generation device 100 performs character recognition using two different optical character recognition devices, thereby making it possible to achieve highly accurate character recognition for images with low accuracy of character recognition.

[0071] More specifically, in the electronic data generation system 10, it is desirable to perform character recognition over a wide range (for example, in units of a page) using the first optical character recognition device 200, which has a low cost of performing character recognition and a low accuracy of character recognition, and to perform character recognition over a narrower range using the second optical character recognition device 300, which has a higher cost of performing character recognition than the first optical character recognition device 200 and a higher accuracy of character recognition (for example, higher accuracy of character recognition on a line-by-line basis) than the first optical character recognition device 200. This makes it possible to increase the accuracy of character recognition and reduce the cost of character recognition in the electronic data generation system 10.

[0072] The display processing unit 110 associates the target image with the text information (character recognition result) identified by the character identification unit 109 and displays them on the screen. An example of the screen will be described below with reference to Fig. 4. Fig. 4 is a diagram showing an example of the screen displayed on the display unit.

[0073] As shown in FIG. 4, screen T10 includes a first display area T11 and a second display area T12. The first display area T11 is an area in which a target image is displayed. The first display area T11 is, for example, a half area on one side of the screen. The second display area T12 is an area in which first text information in the target image and second text information (line information in FIG. 4) in the specific range image are displayed. The second display area T12 is, for example, a half area on the other side of the screen.

[0074] The display processing unit 110, for example, displays characters indicated by line information displayed in the second display area, among characters included in the target image displayed in the first display area T11, in an identifiable manner. Specifically, as shown in FIG. 4, the display processing unit 110, for example, displays an object OT1 that can identify a line image of the target image, and displays an object OT2 in the line information corresponding to the line image. For example, the display color of the object OT1 is the same as the display color of the object OT2. This allows the electronic data generating device 100 to provide the user with the correspondence between the target image and the first text information and the second text information, which are the results of character recognition of a predetermined range of the target image, and the target image, thereby allowing the user to easily grasp erroneous recognition of the target image.

[0075] <<Modifications>> The character specification unit 109 may specify either the character indicated by the first text information or the character indicated by the second text information as a character included in the target image based on the number of inputs, which is the number of times a specific range image is input to the second optical character recognition device 300. Specifically, when the number of inputs of a specific range image (e.g., a line image) input to the second optical character recognition device 300 exceeds a predetermined number, the character specification unit 10 specifies the character indicated by the first text information corresponding to the first accuracy index as a character included in the target image. Thereby, when the cost of character recognition processing by the second optical character recognition device 300 is higher than the cost of character recognition processing by the first optical character recognition device 200, for example, when the cost exceeds a certain cost, the electronic data generation system 10 can achieve cost reduction by performing character recognition using an optical character recognition device with a lower cost.

[0076] In this case, the display processing unit 110 may display the number of inputs, which is the number of times the specific range image has been input to the second optical character recognition device 300, at a predetermined position on the screen T10. Specifically, the display processing unit 110 may display at a predetermined position in the second display area. Furthermore, as shown in FIG. 4, the display processing unit 110 may display the number of inputs in association with the second text information corresponding to the line image ("server" in FIG. 4) that has become the target of input to the second optical character recognition device 300 after the input number has been exceeded ("5 times" in FIG. 4). In this way, the electronic data generation system 10 allows the user to easily grasp the specific range images for which character recognition by the second optical character recognition device 300 could not be performed due to the number of times limit.

[0077] ===First Optical Character Recognition Device 200=== Returning to Fig. 1, a description will be given of the configuration of the first optical character recognition device 200. The first optical character recognition device 200 is a device that, for example, when a target image is input, recognizes characters included in the target image and generates, for example, first text information, a first accuracy index, and coordinates (first generated information) for each recognized character.

[0078] As shown in FIG. 1, the first optical character recognition device 200 includes, for example, a storage unit 210, a transmission / reception unit 220, and a processing unit 230. The storage unit 210 stores various information. The processing unit 230 executes processing for character recognition. The transmission / reception unit 220 transmits and receives various information to and from the electronic data generating device 100. The processing unit 230 analyzes an image using, for example, a neural network trained to distinguish between characters. The neural network includes, for example, a plurality of convolutional network layers and a recurrent network layer. The processing unit 230 segments, for example, a target image into pages, blocks, lines, or characters. By performing character recognition on characters included in the segmented image, the processing unit 230 generates, for example, first generated information for each character. The processing unit 230 may generate, for example, the first generated information of a segmented image (for example, a target image, a block image, or a line image) as a single group.

[0079] ===Second Optical Character Recognition Device 300=== The configuration of the second optical character recognition device 300 will be described with reference to Fig. 1. The second optical character recognition device 300 is a device that, for example, when a specific range image is input, recognizes characters included in the specific range image and generates, for example, second text information, a second accuracy index, and coordinates (second generated information) for each recognized character.

[0080] For example, when the specific range image is a line image, the second optical character recognition device 300 is preferably a device capable of achieving higher accuracy in character identification than that of the first optical character recognition device 200. In this case, in the electronic data generation system 10, by using the second optical character recognition device 300, which has high accuracy in character recognition for line images, for line images for which the accuracy of character recognition in the first optical character recognition device 200 is low, it is possible to improve the accuracy of character recognition.

[0081] As shown in FIG. 1, the second optical character recognition device 300 includes, for example, a storage unit 310, a transmission / reception unit 320, and a processing unit 330. The storage unit 310 stores various information. The processing unit 330 executes processing for character recognition. The transmission / reception unit 320 transmits and receives various information to and from the electronic data generation device 100. The processing unit 330 analyzes an image using, for example, a neural network trained to distinguish between characters. The neural network includes, for example, a plurality of convolutional network layers and a recurrent network layer. The processing unit 330 may be the same as the processing unit 230 of the first optical character recognition device 200, or may be a functional unit that executes processing specialized for character recognition for line images. In this case, the processing unit 330 segments, for example, the line image for each character. Then, the processing unit 330 executes character recognition for the character to generate, for example, second generation information for each character.

[0082] ===User terminal 400=== The configuration of the user terminal 400 will be described with reference to Fig. 1. As shown in Fig. 1, the user terminal 400 includes functional units, for example, a storage unit 410, a transmission / reception unit 420, and a display processing unit 430. Each functional unit is a function realized by, for example, a processor 1001 reading out a program stored in a memory 1002.

[0083] The storage unit 410 stores various types of information. The transmission / reception unit 420 transmits and receives various types of information to and from the electronic data generation device 100. The various types of information acquired by the transmission / reception unit 420 is stored in the storage unit 410. The display processing unit 430 displays a screen T10 acquired from the electronic data generation device 100 on the display unit.

[0084] ===Processing Procedure=== The processing procedure of the electronic data generation system 10 will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a flowchart showing the processing procedure of the electronic data generation system 10. Fig. 6 is a table showing validity indices for line information, which is text information of a one-line character string. In the following, as an example, character recognition for "100BASE-TX switch set", which is a one-line character string included in a target image, will be described.

[0085] In step S100, the electronic data generation device 100 acquires a target image from a predetermined device. The electronic data generation device 100 stores the target image in the storage unit 101. The electronic data generation device 100 transmits the target image to the first optical character recognition device 200.

[0086] In step S101, the first optical character recognition device 200 segments the target image to generate first text information, a first accuracy indicator, and coordinates for each character contained in the target image. The first optical character recognition device 200 transmits the first generated information to the electronic data generating device 100.

[0087] In step S102, the electronic data generation device 100 stores the first generation information for each character in the target image information D101a in association with the target image.

[0088] In step S103, the electronic data generating device 100 determines whether or not the first reference indicator of the character included in the target image is lower than the first reference.

[0089] If it is determined that the first reference indicator is equal to or greater than the first reference (step S103: NO), in step S104, the electronic data generation device 100 identifies the character indicated by the first text information as the character included in the target image.

[0090] If it is determined that the first reference indicator is lower than the first reference (step S103: YES), in step S105, the electronic data generation device 100 refers to the target image information D101a and identifies line information including first text information corresponding to the first reference indicator determined to be lower than the first reference. Specifically, the electronic data generation device 100 identifies the first text information and the first validity indicator shown in FIG. 6(a).

[0091] In step S106, the electronic data generating device 100 identifies a line image (specific range image) corresponding to the line information from the target image based on the coordinates included in the identified line information. The electronic data generating device 100 transmits the identified line image to the second optical character recognition device 300.

[0092] In step S107, the second optical character recognition device 300 segments the line image to generate second text information, a second accuracy indicator, and coordinates for each character contained in the line image. The second optical character recognition device 300 transmits the second generated information to the electronic data generating device 100.

[0093] In step S108, the electronic data generation device 100 stores the second generation information for each character in the specific range image information D101b in association with the specific range image.

[0094] In step S109, the electronic data generation device 100 judges whether the second reference indicators in the specific range image are larger than the second reference. Specifically, it judges whether all the second validity indicators (second reference indicators) of the line information shown in FIG. 6(b) exceed a threshold value (second reference).

[0095] If it is determined that the second reference indicator is equal to or greater than the second reference (step S109: YES), in step S110, the electronic data generation device 100 identifies the character indicated by the second text information included in the line information as the character included in the line image.

[0096] If it is determined that the second reference index is lower than the second reference (step S109: NO), in step S111, the electronic data generating device 100 compares the first accuracy index for the line information corresponding to the specific range image with the second accuracy index. Specifically, the electronic data generating device 100 compares the smallest value of the first accuracy indexes shown in Fig. 6(a) ("0.32" in Fig. 6(a)) with the smallest value of the second accuracy indexes shown in Fig. 6(b) ("0.57" in Fig. 6(b)).

[0097] In step S112, if the electronic data generation device 100 determines that the smallest value of the first accuracy indicators is greater than the smallest value of the second accuracy indicators, it identifies the line information including the first text information (line information in FIG. 6(a)) as a character included in the line image. On the other hand, if the electronic data generation device 100 determines that the smallest value of the second accuracy indicators is greater than the smallest value of the first accuracy indicators, it identifies the line information including the second text information (line information in FIG. 6(b)) as a character included in the line image.

[0098] This makes it possible to adopt a character string that is estimated to have been recognized more accurately between a predetermined range of character strings including characters with low accuracy in the first character recognition by the first optical character recognition device 200 and a character string in the second character recognition by the second optical character recognition device 300.

[0099] In step S112, the electronic data generating device 100 may compare the average value of the first accuracy index ("Average value" in FIG. 6(a)) with the average value of the second accuracy index ("Average value" in FIG. 6(b)). In this case, the electronic data generating device 100 identifies the line information (line information in FIG. 6(b)) corresponding to the larger average value as a character included in the line image.

[0100] In step S113, the electronic data generation device 100 causes the display unit of the user terminal 400 to display a screen T10 shown in FIG. 4, which allows a comparison between the target image and the result of character recognition of the target image.

[0101] When a plurality of pieces of line information are specified in step S106, the electronic data generation system 10 repeatedly executes steps S106 to S112 the number of times corresponding to the number of pieces of line information specified.

[0102] In this way, in the electronic data generation system 10, it is desirable to perform character recognition by the second optical character recognition device 300, which can perform character recognition by line units, more appropriately for lines determined to have low accuracy of character recognition as a result of character recognition by the first optical character recognition device 200, which can perform character recognition by page units more appropriately. Then, the electronic data generation system 10 compares the character recognition result by the first optical character recognition device 200 with the character recognition result by the second optical character recognition device 300, and adopts the character recognition result with higher accuracy. In other words, in the electronic data generation device 100, by performing character recognition on different character recognition ranges by two different optical character recognition devices, it becomes possible to realize highly accurate character recognition for images with low accuracy of character recognition.

[0103] ===Hardware Configuration=== An example of a hardware configuration for implementing the electronic data generation device 100, the first optical character recognition device 200, the second optical character recognition device 300, and the user terminal 400 on a computer will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of a hardware configuration of a computer.

[0104] As shown in FIG. 7, a computer 1000 includes a processor 1001, a memory 1002, a storage device 1003, an input I / F unit 1004, a data I / F unit 1005, a communication I / F unit 1006, and a display unit 1007.

[0105] The processor 1001 is a control unit that controls various processes in the computer 1000 by executing programs stored in the memory 1002 .

[0106] The memory 1002 is a storage medium such as a RAM (Random Access Memory), etc. The memory 1002 temporarily stores the program code of the program executed by the processor 1001 and data required when the program is executed.

[0107] The storage device 1003 is a non-volatile storage medium such as a hard disk drive (HDD), a flash memory, etc. The storage device 1003 stores an operating system and various programs for implementing the above-mentioned components.

[0108] The input I / F unit 1004 is a device for receiving input from a user. Specific examples of the input I / F unit 1004 include a keyboard, a mouse, a touch panel, various sensors, a wearable device, etc. The input I / F unit 1004 may be connected to the computer 1000 via an interface such as a USB (Universal Serial Bus).

[0109] The data I / F unit 1005 is a device for inputting data from outside the computer 1000. A specific example of the data I / F unit 1005 is a drive device for reading data stored in various storage media. The data I / F unit 1005 may be provided outside the computer 1000. In that case, the data I / F unit 1005 is connected to the computer 1000 via an interface such as a USB.

[0110] The communication I / F unit 1006 is a device for performing data communication via the Internet N, either wired or wirelessly, with an external device of the computer 1000. The communication I / F unit 1006 may be provided outside the computer 1000. In that case, the communication I / F unit 1006 is connected to the computer 1000 via an interface such as a USB.

[0111] The display unit 1007 is a device for displaying various types of information. Specific examples of the display unit 1007 include a liquid crystal display, an organic EL (Electro-Luminescence) display, a display of a wearable device, and the like. The display unit 1007 may be provided outside the computer 1000. In that case, the display unit 1007 is connected to the computer 1000 via, for example, a display cable. In addition, when a touch panel is adopted as the input I / F unit 1004, the display unit 1007 can be configured integrally with the input I / F unit 1004.

[0112] ===Summary=== <1> The electronic data generation system 10 in this embodiment includes a target image acquisition unit 102 that acquires a target image including characters from a predetermined device, a first information acquisition unit 103 that acquires, from a first optical character recognition device 200 that recognizes each of a plurality of characters included in the target image as first text information, first text information for each of the plurality of characters and a first accuracy index indicating the degree of recognition accuracy for the character indicated by the first text information, and a first reference index based on at least one of the first accuracy indexes for the plurality of characters, the first information acquisition unit 103 acquiring, from the first optical character recognition device 200 that recognizes each of the plurality of characters included in the target image as first text information, a first reference index indicating the degree of recognition accuracy for the character included in the image. When the first optical character recognition device 200 determines that the first reference index is lower than the first reference, the second information acquisition unit 107 acquires second text information about the characters included in the specific range image from a second optical character recognition device 300 different from the first optical character recognition device 200, which is a specific range image that is an image of a part of the range of the target image including a character corresponding to the first reference index determined to be lower than the first reference, and recognizes each of at least one character included in the specific range image as second text information, and a character identification unit 109 identifies the characters included in the target image based on the first text information and the second text information. As a result, the electronic data generation system 10 can realize highly accurate character recognition by performing character recognition on an image with low accuracy of character recognition using a plurality of optical character recognition devices with different target ranges for character recognition (for example, the first optical character recognition device 200 performs character recognition on a page basis and the second optical character recognition device 300 performs character recognition on a line basis).

[0113] <2> Further, the second information acquisition unit 107 in the electronic data generation system 10 in this embodiment acquires from the second optical character recognition device 300 second text information for at least one character included in the specific range image and a second accuracy index indicating the degree of accuracy of recognition for the character indicated by the second text information, and the character identification unit 109 identifies the character included in the specific range image based on the result of determining the magnitude relationship between the second reference index based on at least one of the second accuracy indexes for each of the at least one character included in the specific range image and the second reference regarding the accuracy of recognition of the character included in the image. This enables the electronic data generation system 10 to appropriately recognize a character string in the second character recognition by the second optical character recognition device 300 for a character string in a predetermined range including a character with low accuracy in the first character recognition by the first optical character recognition device 200.

[0114] <3> Furthermore, when it is determined that the second reference index for all characters included in the specific range image is equal to or greater than the second reference, the character identification unit 109 of the electronic data generation system 10 in this embodiment identifies the characters indicated by the second text information corresponding to the second accuracy index as the characters included in the specific range image. This allows the electronic data generation system 10 to properly recognize character strings in the second character recognition by the second optical character recognition device 300 for character strings in a predetermined range including characters with low accuracy in the first character recognition by the first optical character recognition device 200.

[0115] <4> Furthermore, when the character identification unit 109 of the electronic data generation system 10 in this embodiment determines that the second reference index for at least one of the characters included in the specific range image is lower than the second reference, it identifies the characters included in the specific range image based on the result of determining the magnitude relationship between the first accuracy index of at least one character included in the range corresponding to the specific range image in the target image and the second accuracy index of at least one character included in the specific range image. This allows the electronic data generation system 10 to realize more appropriate character recognition using a character string that is estimated to have been recognized more accurately between a character string in a predetermined range including a character with low accuracy in the first character recognition by the first optical character recognition device 200 and a character string in the second character recognition by the second optical character recognition device 300.

[0116] <5> Furthermore, the target image acquisition unit 102 of the electronic data generation system 10 in this embodiment acquires a target image that is an image in units of one page, and the information transmission unit 106 transmits a specific range image that is an image in units of lines of text included in the target image to the second optical character recognition device 300. As a result, in the electronic data generation system 10, for example, for a line image in which the accuracy of character recognition in the first optical character recognition device 200 is low, by using the second optical character recognition device 300, which has high accuracy of character recognition for line images, it becomes possible to improve the accuracy of character recognition.

[0117] <6> Furthermore, the character identification unit 109 of the electronic data generation system 10 in this embodiment identifies a character indicated by the first text information corresponding to the first accuracy index as a character included in the specific range image when inputting the specific range image to the second optical character recognition device 300 satisfies a condition related to inputting the specific range image to the second optical character recognition device 300. As a result, when the cost of character recognition processing by the second optical character recognition device 300 is higher than the cost of character recognition processing by the first optical character recognition device 200, for example, when the cost exceeds a certain cost, the electronic data generation system 10 can achieve cost reduction by performing character recognition using an optical character recognition device with a lower cost.

[0118] <7> Moreover, the electronic data generation system 10 in this embodiment further includes a display processing unit 110 that displays the target image in a first display area T11 of the screen T10 and displays second text information indicating characters included in the specific range image in a second display area T12 different from the first display area T11 on the screen T10. This allows the electronic data generation system 10 to provide the user with the correspondence between the target image and the first text information and the second text information, which are the results of character recognition of a predetermined range of the target image, so that the user can easily grasp misrecognition of the target image, etc.

[0119] <8> Moreover, the display processing unit 110 in the electronic data generation system 10 in this embodiment displays the second text information indicating the characters included in the specific range image and the first text information indicating the characters included in the target image excluding the characters included in the specific range image in the second display area T12, and displays the second text information in an identifiable manner. This allows the electronic data generation system 10 to display the second text information in the second optical character recognition device 300 in a manner that is easily identifiable by the user, thereby allowing the user to easily grasp the degree of misrecognition of the target image, etc.

[0120] <9> Furthermore, the display processing unit 110 of the electronic data generation system 10 in this embodiment displays, in a predetermined display area of ​​the screen, the number of inputs, which is the number of times the specific range image has been input to the second optical character recognition device 300. This allows the electronic data generation system 10 to allow the user to easily grasp the number of times character recognition has been performed by the second optical character recognition device 300. [Explanation of symbols]

[0121] 10...electronic data generation system, 100...electronic data generation device, 101...memory unit, 102...target image acquisition unit, 103...first information acquisition unit, 104...first judgment unit, 105...specific range identification unit, 106...information transmission unit, 107...second information acquisition unit, 108...second judgment unit, 109...character identification unit, 110...display processing unit, 200...first optical character recognition device, 300...second optical character recognition device, 400...user terminal.

Claims

1. a target image acquisition unit that acquires a target image including characters from a predetermined device; a first information acquisition unit that acquires, from a first optical character recognition device that recognizes each of a plurality of characters included in the target image as first text information, the first text information for each of the plurality of characters and a first accuracy index that indicates a degree of recognition accuracy for the character indicated by the first text information; a second information acquisition unit that acquires, when a first reference indicator based on at least one of the first accuracy indicators for the plurality of characters is determined to be lower than a first standard related to accuracy of recognition of characters included in an image, from a second optical character recognition device different from the first optical character recognition device, which is a specific range image that is an image of a partial range of the target image including characters corresponding to the first reference indicator determined to be lower than the first standard, and which recognizes each of at least one character included in the specific range image as second text information, the second text information for each of at least one character included in the specific range image and a second accuracy indicator indicating a degree of accuracy of recognition for the character indicated by the second text information; a character identification unit that identifies a character included in the specific range image based on a result of determining whether a second reference indicator based on at least one of the second accuracy indicators for each of at least one character included in the specific range image is larger than a second reference indicator related to accuracy of recognition of the character included in the image; and Equipped with when it is determined that the second reference index for at least one character among the characters included in the specific range image is lower than the second reference, the character identification unit identifies a character included in the specific range image based on a result of determining a magnitude relationship between the first accuracy index of at least one character included in a range corresponding to the specific range image in the target image and the second accuracy index of at least one character included in the specific range image. Information processing system.

2. a target image acquisition unit that acquires a target image including characters from a predetermined device; a first information acquisition unit that acquires, from a first optical character recognition device that recognizes each of a plurality of characters included in the target image as first text information, the first text information for each of the plurality of characters and a first accuracy index that indicates a degree of recognition accuracy for the character indicated by the first text information; a second information acquisition unit that acquires the second text information about characters included in a specific range image from a second optical character recognition device different from the first optical character recognition device, the second information acquisition unit being configured to acquire, when a first reference indicator based on at least one of the first accuracy indicators for the plurality of characters is determined to be lower than a first standard related to the accuracy of recognition of characters included in an image, a specific range image that is an image of a partial range of the target image and includes characters corresponding to the first reference indicator determined to be lower than the first standard, the specific range image being input and recognizing each of at least one character included in the specific range image as second text information; a character identification unit that identifies a character included in the target image based on the first text information and the second text information; Equipped with the character identification unit, when the number of times that the specific range image is input to the second optical character recognition device exceeds a predetermined number of times, identifies a character indicated by the first text information corresponding to the first accuracy index as a character included in the specific range image. Information processing system.

3. when it is determined that the second reference indicator for all characters included in the specific range image is equal to or greater than the second reference, the character identification unit identifies a character indicated by the second text information corresponding to the second accuracy indicator as a character included in the specific range image. The information processing system according to claim 1 .

4. The target image acquisition unit acquires the target image, which is an image of one page unit, the second information acquisition unit acquires, from the second optical character recognition device, the second text information about characters included in the specific range image, which is an image of a line of a sentence included in the target image; The information processing system according to claim 1 .

5. Displaying the target image in a first display area of ​​a screen; a display processing unit that displays the second text information indicating characters included in the specific range image in a second display area different from the first display area on the screen; The information processing system according to claim 1 further comprising:

6. The display processing unit is displaying, in the second display area, the second text information indicating characters included in the specific range image and the first text information indicating characters included in the target image excluding the characters included in the specific range image; displaying the second text information in an identifiable manner; 6. The information processing system according to claim 5.

7. the display processing unit displays, in a predetermined display area of ​​the screen, an input count, which is the number of times the specific range image has been input to the second optical character recognition device; 6. The information processing system according to claim 5.

8. The computer Acquiring a target image including text from a predetermined device; acquiring, from a first optical character recognition device that recognizes each of a plurality of characters included in the target image as first text information, the first text information for each of the plurality of characters and a first accuracy index indicating a degree of recognition accuracy for the character indicated by the first text information; When a first reference indicator based on at least one of the first accuracy indicators for the plurality of characters is determined to be lower than a first standard regarding accuracy of recognition of characters included in an image, a specific range image is an image of a partial range of the target image including characters corresponding to the first reference indicator determined to be lower than the first standard, and each of at least one character included in the specific range image is recognized as second text information by a second optical character recognition device different from the first optical character recognition device, and the second text information and a second accuracy indicator indicating a degree of accuracy of recognition for the character indicated by the second text information are obtained from the second optical character recognition device; identifying a character included in the specific range image based on a result of determining whether a second reference indicator based on at least one of the second accuracy indicators for each of at least one character included in the specific range image is larger than a second reference indicator related to accuracy of recognition of the character included in the image; when it is determined that the second reference index for at least one character among the characters included in the specific range image is lower than the second reference, identifying characters included in the specific range image based on a result of determining a magnitude relationship between the first accuracy index of at least one character included in a range corresponding to the specific range image in the target image and the second accuracy index of at least one character included in the specific range image; An information processing method for performing the above.

9. On the computer, Acquiring a target image including text from a predetermined device; acquiring, from a first optical character recognition device that recognizes each of a plurality of characters included in the target image as first text information, the first text information for each of the plurality of characters and a first accuracy index indicating a degree of recognition accuracy for the character indicated by the first text information; When a first reference indicator based on at least one of the first accuracy indicators for the plurality of characters is determined to be lower than a first standard regarding accuracy of recognition of characters included in an image, a specific range image is an image of a partial range of the target image including characters corresponding to the first reference indicator determined to be lower than the first standard, and each of at least one character included in the specific range image is recognized as second text information by a second optical character recognition device different from the first optical character recognition device, and the second text information and a second accuracy indicator indicating a degree of accuracy of recognition for the character indicated by the second text information are obtained from the second optical character recognition device; identifying a character included in the specific range image based on a result of determining whether a second reference indicator based on at least one of the second accuracy indicators for each of at least one character included in the specific range image is larger than a second reference indicator related to accuracy of recognition of the character included in the image; when it is determined that the second reference index for at least one character among the characters included in the specific range image is lower than the second reference, identifying characters included in the specific range image based on a result of determining a magnitude relationship between the first accuracy index of at least one character included in a range corresponding to the specific range image in the target image and the second accuracy index of at least one character included in the specific range image; A program that executes the following.

Images