Image reading device

Abstract

To classify original images on a document basis only by reading an original bundle.SOLUTION: A user selects a document dividing method. An image reading section reads an original bundle, acquires an original image and determines a head page of the original image in accordance with the selected dividing method. A page number recognition section extracts a page number from the original image by executing page number recognition processing and determines an original image indicating a first page as a head page of a document. A layout recognition section detects a margin area or a background color from the original image by executing layout recognition processing and determines a head page of a document. A title recognition section extracts a title by executing title recognition processing and determines a head page of a document. On the basis of the determined head pages, a division section divides the original image on a document basis, and the divided original images are filed and stored in a storage section.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an image reading device that reads a document stack including a plurality of types of documents and divides the document images into documents. [Background technology]

[0002] Patent Document 1 describes a method for dividing multiple document data into documents based on history information related to the execution history of a workflow. The history information includes at least one of the following information: document layout, font, creation date, title, footer, background image, watermark, and logo. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-100432 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technique described in Patent Document 1 has a problem in that multiple documents cannot be divided unless there is a workflow execution history.

[0005] SUMMARY OF THE PRESENT EMBODIMENT An object of the present invention is to provide an information processing apparatus and an image reading apparatus that can divide an original image into documents simply by reading a stack of originals. [Means for solving the problem]

[0006] An image reading device according to one aspect of the present invention includes an image reading unit that acquires document images of multiple pages by reading a stack of documents including multiple documents one by one, a page number recognition unit that recognizes the page number of each document image of the multiple pages acquired by the image reading unit and determines the first page of the document, a layout recognition unit that recognizes a margin area width and a background color of each document image and determines the first page of the document, a title recognition unit that recognizes a title from each document image and determines the first page of the document, a control unit that causes the page number recognition unit, the layout recognition unit, or the title recognition unit to determine whether each document image is the first page of the document, and a division unit that divides the document images into documents of the same type using a result of the determination by the page number recognition unit, the layout recognition unit, or the title recognition unit that performed the determination as to whether each document image is the first page of the document. Effect of the Invention

[0007] According to the present invention, an original image can be easily divided into documents, which can reduce the time and effort required to manually classify the originals and then have the originals read by a reading device. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram showing an electrical configuration of an image reading apparatus according to an embodiment of the present invention; [Diagram 2] 13 is a flowchart illustrating a flow of a document division process. [Diagram 3] 11 is a flowchart showing the flow of a page number recognition process. [Figure 4] 11 is a flowchart showing the flow of a layout recognition process. [Diagram 5] 13 is a flowchart showing the flow of a title recognition process. [Figure 6] FIG. 11 is a diagram for explaining division of a document image. [Figure 7] FIG. 11 is a diagram for explaining division of a document image. [Figure 8] FIG. 11 is a diagram for explaining division of a document image. [Figure 9] FIG. 11 is a diagram for explaining division of a document image. [Figure 10] FIG. 11 is a diagram for explaining division of a document image. [Figure 11] FIG. 11 is a diagram for explaining division of a document image. [Figure 12] FIG. 11 is a diagram for explaining division of a document image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] An information processing device and an image reading device according to an embodiment of the present invention will be described below with reference to the drawings. Note that in this embodiment, an image reading device equipped with the information processing device of the present invention will be described as an example. Fig. 1 is a diagram showing the electrical configuration of an image reading device according to an embodiment of the present invention.

[0010] The image reading device 1 is configured to include a control unit 10, an input receiving section 12, an image reading section 13, a storage section 14, and a communication control section 15. The input receiving section 12 includes hardware keys such as a decision key for confirming various operations and settings and a start key, and a display section 121. The display section 121 displays an operation screen, messages, and the like. The display section 121 may be configured integrally with a touch panel.

[0011] The image reading unit 13 includes, for example, a scanner, and reads an image of a document to obtain the document image. The image reading unit 13 also includes an automatic document feeder (not shown), and can continuously read a stack of documents consisting of multiple documents. The communication control unit 15 includes a communication module and the like, and transmits and receives various data to and from external devices via a network.

[0012] The storage unit 14 is a large-capacity storage device consisting of an SSD or HDD that stores image data, various programs, data tables, etc. The storage unit 14 stores a machine learning model 141 for page number extraction, a machine learning model 142 for layout recognition, and a machine learning model 143 for title recognition.

[0013] The machine learning model for page extraction has been trained to extract page number candidates from the character region of a manuscript image. The machine learning model for layout recognition 142 has been trained to determine whether the most recently read manuscript image is the same document as the manuscript image one page before, based on the margin region width and background color of the manuscript image. The machine learning model for title recognition 143 has been trained to extract title candidates from the character region of a manuscript image.

[0014] The control unit 10 is composed of a processor, a RAM (Random Access Memory), a ROM (Read Only Memory), etc. The processor is a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), etc. The control unit 10 functions as a control unit 111, a page number recognition unit 112, a layout recognition unit 113, a title recognition unit 114, and a division unit 115 by the processor executing a control program stored in the ROM, etc. Note that each of the above-mentioned parts of the control unit 10 may be composed of a hardware circuit, regardless of the operation based on the control program.

[0015] The control unit 111 is responsible for overall operational control of the image reading device 1. The page number recognition unit 112 recognizes the page number of each document image. The layout recognition unit 113 detects the margin area width and background color of each document image. The title recognition unit 114 extracts the title of each document image. The division unit 115 performs document division processing to divide the document images of the document stack read by the image reading unit 13.

[0016] Here, the document division process will be described. FIG. 2 is a flowchart showing the flow of the document division process. First, the user operates the input reception unit 12 to input a selection instruction to select one of the document division methods. The control unit 111 receives the selection instruction (S11). The document division methods are a method using the page number of the document image, a method using the margin area width and background color of the document image, and a method using the title included in the document image. The user operates the input reception unit 12 to input a selection instruction to select one or more methods, and the control unit 111 receives the selection instruction.

[0017] A selection instruction by the user to specify a document division method may be received by control unit 111 every time the user scans a stack of documents. Alternatively, an administrator of image reading device 1 may operate input receiving unit 12 to input one of the methods as an initial value, and control unit 111 may execute the document division method indicated by the initial value as a default. In this case, control unit 111 may change the default document division method in response to an instruction from the user input to input receiving unit 12.

[0018] Then, under the control of control unit 111, image reading unit 13 reads the document stack to obtain document images (S12). Control unit 111 temporarily stores each document image obtained by document reading by image reading unit 13 together with information indicating the reading order in a predetermined storage area, for example, a non-volatile memory or the like built into control unit 10. Thereafter, control unit 111 executes document division processing (S14, S15, S16) according to the document division method indicated by the selection instruction (S13).

[0019] Next, the page number recognition process (process S14 in the flowchart of FIG. 2) will be described. FIG. 3 is a flowchart showing the flow of the page number recognition process. First, the page number recognition unit 112 detects position information of a character area from the document image read by the image reading unit 13 using optical character recognition technology (S21). The page number recognition unit 112 recognizes characters from the document image read by the image reading unit 13 using, for example, optical character recognition processing technology, extracts a character area consisting of a group of recognized characters, and obtains position information of the extracted character area. Specifically, the page number recognition unit 112 extracts a character area from the document image, and obtains the coordinates of four points (topmost position of the left end, topmost position of the right end, below the bottommost position of the left end, and bottommost position of the right end) of the extracted character area as position information.

[0020] Next, the page number recognition unit 112 reads the machine learning model 141 for extracting page numbers (S22), and uses position information of the character area of ​​the document image, etc. to identify a position where an image indicating a page number is assumed to exist in the character area of ​​the document image, and determines whether an image indicating a page number exists from the area at the identified position. If the page number recognition unit 112 determines that an image indicating a page number exists, it extracts a character area including an image indicating a page number from the document image (S23). Furthermore, the page number recognition unit 112 extracts images indicating multiple types of page numbers as images indicating page numbers using the machine learning model 141 for extracting page numbers. For example, the page number recognition unit 112 extracts images indicating page numbers such as "-1-", "page 1", and "p.1" as images indicating different types of page numbers using the machine learning model 141 for extracting page numbers.

[0021] Furthermore, the page number recognition unit 112 may store page characters (such as "Page 1") that are predetermined to indicate page numbers, detect whether or not there is a character located at the page character in the recognized character group, and if the character is present, determine that the character area includes an image indicating a page number and extract the character area as a character area including an image indicating a page number.

[0022] When the page number recognition unit 112 extracts a character area of ​​an image showing a page number (S23; YES), it determines the document image including the page number as the document image that is the first page of the document (S25). However, when the page number recognition unit 112 does not extract a character area that is thought to be a page number (S23; NO), the process ends without determining that the document image is the first page of the document.

[0023] The page number recognition unit 112 executes the above-mentioned processes S21 to S25 for all document images read by the image reading unit 13. When this process ends, the control unit 111 advances the process to process S17 of the document division process.

[0024] If the page number cannot be extracted in step S22 (S23; NO), the page number recognition section 112 ends the page number recognition process, and the control section 111 advances the process to step S17 of the document division process.

[0025] Next, the layout recognition process (process S15 in the flowchart of FIG. 2) will be described. FIG. 4 is a flowchart showing the flow of the layout recognition process. First, the layout recognition unit 113 detects the margin area of ​​the document image (S31). The margin area detection method uses known technology. Next, the layout recognition unit 113 detects the background color of the document image (S32). For example, the layout recognition unit 113 detects the luminance (pixel value) of a 2 mm-wide area at the leading edge of the document image, calculates the average value, and sets it as the luminance of the background color. The layout recognition unit 113 stores the margin area of ​​the document image and a numerical value indicating the luminance of the background color for each page of the document image read.

[0026] Then, the layout recognition unit 113 reads the machine learning model for layout recognition 142 (S33). The layout recognition unit 113 uses the machine learning model for layout recognition 142 to determine whether the latest document image and the document image one page before are the same document, based on the blank areas of the document images of the latest document image and the document image one page before that are read at this time and a numerical value indicating the brightness of the background color (S34). The layout recognition unit 113 compares the blank areas of the document images of the latest document image and the document image one page before that with a numerical value indicating the brightness of the background color, and determines that the latest document image and the document image one page before are the same document if the degree of match is a predetermined value (e.g., 95%) or more.

[0027] Here, if the layout recognition unit 113 determines that the latest original image and the original image of the previous page are different documents, it determines that the latest original image is the first page of that document. Also, if the layout recognition unit 113 determines that the latest original image and the original image of the previous page are the same document, it determines that the latest original image is the original image that is the next page of the original image of the previous page. Note that the layout recognition unit 113 executes processes S31 to S34 for all original images. When the processes for all original images are completed, the control unit 111 proceeds to process S17 of the document division process.

[0028] Next, the title recognition process (process S16 in the flowchart of FIG. 2) will be described. FIG. 5 is a flowchart showing the flow of the title recognition process. First, the title recognition unit 114 detects position information of the character area from the document image using optical character recognition technology (S41). The position information of the character area is, for example, the coordinates of the four points of the acquired character area (topmost position of the left edge, topmost position of the right edge, below the bottommost position of the left edge, and bottommost position of the right edge).

[0029] Next, the title recognition unit 114 reads the title recognition machine learning model 143 (S42), and extracts a character area that is a title candidate from the character area of ​​the manuscript image using the title recognition machine learning model 143. If the title recognition unit 114 extracts a character area that is a title candidate from the manuscript image (S43; YES), it determines that the manuscript image is the first page of the document (S44).

[0030] The title recognition section 114 executes the processes S41 to S44 for all the document images read by the image reading section 13. When the processes for all the document images are completed, the control section 111 advances the process to the document division process S17.

[0031] In order to improve the accuracy of the title determination, the title recognition unit 114 may convert the character area of ​​the title candidate extracted by the title recognition unit 114 into text using optical character recognition technology. In this case, the storage unit 14 stores in advance words that are title candidates (for example, text indicating "bill", "Invoice", etc.). The title recognition unit 114 compares the text of the character area with the title candidates stored in the storage unit 14. If there is text in the text of the character area that matches the title candidates stored in the storage unit 14, the title recognition unit 114 determines that the text is the title.

[0032] Returning to the description of the document division process shown in the flowchart of Fig. 2, based on the results of the determination of the first page performed in each of the processes S14, S15, and S16, the division unit 115 divides the document image of the document stack read in process S12 (S17).

[0033] 6 to 8 are diagrams for explaining division of an original image. Fig. 6 is a diagram for explaining a case where an original image is divided based on page number recognition performed by page number recognition unit 112, and D11 to D16 are examples of original images. Here, it is assumed that page number recognition unit 112 determines that the original image of the original first read by image reading unit 13 is the first page of the document.

[0034] When the page number recognition unit 112 extracts page numbers from the document images D11 to D16 and extracts page numbers indicating the first page from the document images D11 and D14, the page number recognition unit 112 determines that the document images D11 and D14 are the first pages of the document. When there are two document images that are the first pages, the division unit 115 determines that the document image group consisting of the document images of the multiple documents read by the image reading unit 13 is divided into two documents, and assigns each document image to one of the two documents. Since the control unit 111 stores the reading order of each document image obtained by document reading by the image reading unit 13, the division unit 115 classifies and divides the document image of the next reading order following the document image of the determined first page according to the reading order as the document image of the next page of the document image of the first page, into the same document as the document image of the first page. When multiple types of page numbers are extracted by the page number recognition unit 112, the division unit 115 performs the division process for each different type of page number.

[0035] The dividing unit 115 classifies and divides the document image from which the page number is extracted first in the reading order as the first page, and subsequent document images from which the same type of page number is extracted as document images of pages following the document image from which the page number is extracted first.

[0036] 6, the division unit 115 divides the document images D11 to D13 from which the same type of page numbers are extracted and the document images D14 to D16 from which different types of page numbers are extracted into separate documents, and stores each document as separate files (document 11 and document 12) in the storage unit 14 (S18). Alternatively, the communication control unit 15 transmits document 11 and document 12 as separate files to an external terminal designated by the user.

[0037] 7 is a diagram illustrating a case where an original image is divided based on the layout recognition performed by the layout recognition unit 113. As a result of the layout recognition unit 113 detecting the margin area and background color from the original images D21 to D26, the layout recognition unit 113 determines that the original image D21 and the original image D24 are the first page of the document. The original image D24 has the same background color as the original image D23 of the previous page, but has a different width of the margin area, so the layout recognition unit 113 determines that the original image D24 is the first page of the document. Since the control unit 111 stores the reading order of each original image obtained by the original reading by the image reading unit 13, the dividing unit 115 classifies and divides the original image of the next reading order following the determined original image of the first page according to the reading order as the original image of the next page of the original image of the first page, into the same document as the original image of the first page.

[0038] Therefore, the division unit 115 divides the document images D21 to D23 and the document images D24 to D26 into different documents, stores them as different files (document 21 and document 22) in the storage unit 14 (S18). Alternatively, the communication control unit 15 transmits the documents 21 and 22 as different files to an external terminal designated by the user.

[0039] 8 is a diagram illustrating another example of a case where a document image is divided based on the layout recognition performed by the layout recognition unit 113. As a result of the layout recognition unit 113 detecting the margin area and background color from the document images D21 to D26, the layout recognition unit 113 determines that the document images D21, D24, and D26 are the first page of the document. The document image 24 has the same background color as the document image D23 of the previous page but has a different width of the margin area, so the layout recognition unit 113 determines that the document image D24 is the first page of the document, and the document image 26 has the same margin area as the document image D25 of the previous page but has a different background color, so the layout recognition unit 113 determines that the document image D26 is the first page of the document. In this case, too, the control unit 111 stores the reading order of each document image obtained by document reading by the image reading unit 13, so the division unit 115 classifies and divides the document image of the next reading order following the determined first page document image in accordance with the reading order as the document image of the next page following the first page document image, into the same document as the first page document image.

[0040] Therefore, the division unit 115 divides the document images D21 to D23, the document images D24 and D25, and the document image D26 into different documents, stores them as different files (document 21, document 22, and document 23) in the storage unit 14 (S18). Alternatively, the communication control unit 15 transmits the documents 21, 22, and 23 as different files to an external terminal designated by the user.

[0041] As described above, according to the above embodiment, when the image reading unit 13 reads a stack of documents including multiple types of documents, the document images can be easily classified by document and saved as image data. Therefore, classification is possible without having to manually classify the documents and then have the reading device read them. In addition, by classifying the documents according to the characteristics of each document such as page number extraction, layout recognition, title recognition, etc., the presence or absence of pages in the document, the presence or absence of a title, or the margin area and background color, it is possible to efficiently and accurately determine whether each document image obtained by reading should be the same document, and easily classify the documents into the same document or different documents according to the contents of the documents.

[0042] In addition, the user can select from among methods using the page number of the original image to determine whether or not each original image is the first page, a method using the margin area and background color of the original image, or a method using the title included in the original image, so that the above division can be performed in accordance with the characteristics of the original image according to the user's needs.

[0043] The document division method may be a combination of page number recognition, layout recognition, and title recognition. In this case, the page number recognition unit 112, layout recognition unit 113, and title recognition unit 114 perform the determination for all document images obtained by reading. Fig. 9 is a diagram for explaining a case where a document image is divided by combining page number recognition, layout recognition, and title recognition.

[0044] A case will be described in which document images D31 to D36 having contents as shown in FIG. 9 are obtained by image reading unit 13 reading a plurality of documents.

[0045] In this case, the layout recognition unit 113 determines that the blank areas and background colors are the same for the original images D31 to D33. The title recognition unit 114 detects a title candidate of "Invoice" from the original image D33. The layout recognition unit 113 determines that the blank areas and background colors are the same for the original images D34 to D36. Furthermore, as a result of extracting page numbers from the original images D31 to D36, the page number recognition unit 112 extracts the page number indicating the first page from the original image D35 and the page number indicating the second page from the original image D36.

[0046] At this time, the page number recognition unit 112 extracts the page numbers from the document images D31 to D36 and determines that the document image D35 is the first page of the document.

[0047] Furthermore, as a result of the layout recognition unit 113 detecting the margin areas and background colors from the original images D31 to D36, the layout recognition unit 113 determines that the original image D34 is the first page of the document because the background color of the original image D34 is different from that of the previous page, the original image D33.

[0048] Furthermore, when the title recognition unit 114 extracts title candidates from the document images D31 to D36 and detects the title candidate "Invoice" from the document image D33, the title recognition unit 114 determines that the document image D33 is the first page of the document.

[0049] In this case too, the division unit 115 classifies and divides the document image of the next reading order following the document image of the determined first page as the document image of the next page following the document image of the first page in accordance with the reading order, into the same document as the document image of the first page.

[0050] From this result, the division unit 115 divides the original images D31 to D36 into document 31 (original images D31 and D32), document 32 (original image D33), document 33 (original image D34), and document 34 (original images D35 and D36), and stores each of them as a separate file in the storage unit 14 (S18). Alternatively, the communication control unit 15 transmits each of the divided documents as a different file to an external terminal designated by the user. By combining document division methods in this way, the original images can be classified by document with even greater accuracy.

[0051] Furthermore, if the page number recognition unit 112 extracts images indicating multiple types of page numbers as images indicating page numbers using the machine learning model 141 for page number extraction, and further determines the page order among page numbers of the same type and extracts images indicating page numbers (for example, even recognizing numbers indicating page numbers), for example, when multiple types of manuscript documents 51 to 54 shown in FIG. 10 are mixed and placed on the image reading unit 13 with each page of each document and the image reading unit 13 is caused to read the mixed manuscript, and the manuscript images of each manuscript are read in the order shown in FIG. 11, it becomes possible to rearrange the manuscript images according to the page order expected for each of the original multiple manuscript documents as shown in FIG. 12, and then classify the documents into each of the manuscript images of each manuscript document. [Explanation of symbols]

[0052] 1 Image reader 10. Control Unit 111 Control section 112 Page Number Recognition Unit 113 Layout Recognition Unit 114 Title Recognition Section 115 Split section 12 Input reception section 121 Display section 13 Image reading unit 14 Storage section 141 Machine learning model for page number extraction 142 Machine Learning Model for Layout Recognition 143 Machine learning model for title recognition 15 Communication control section

Claims

1. an image reading unit that reads a document stack including a plurality of documents one by one to obtain document images of a plurality of pages; a page number recognition unit that recognizes a page number of each of the document images of the plurality of pages acquired by the image reading unit and determines a first page of the document; a layout recognition unit that recognizes a margin area width and a background color of each of the document images to determine a first page of the document; a title recognition unit that recognizes a title from each of the document images and determines a first page of the document; a control unit that causes the page number recognition unit, the layout recognition unit, or the title recognition unit to determine whether each of the document images is a first page of the document; and a division unit that divides the original images into documents of the same type using the result of the determination by either the page number recognition unit, the layout recognition unit, or the title recognition unit that performed the determination as to whether each of the original images is the first page of the document.

2. the control unit causes the page number recognition unit, the layout recognition unit, and the title recognition unit to determine whether each of the document images is a first page of the document; 2. The image reading device according to claim 1, wherein the dividing unit divides the original images into documents of the same type using the results of determination by the page number recognition unit, the layout recognition unit, and the title recognition unit as to whether each of the original images is the first page of the document.

3. the page number recognition unit extracts a plurality of types of page numbers as the page numbers, and recognizes the page numbers by determining the page order among the page numbers of the same type; 3. The image reading device according to claim 2, wherein the dividing unit divides the original images into documents of the same type using a determination result of whether each original image is the first page of the document after extracting multiple types of page numbers by the page number recognition unit and determining the page order among page numbers of the same type, and a determination result of whether each original image is the first page of the document by the layout recognition unit and the title recognition unit.

4. an input receiving unit that receives a selection input from among a method of determining whether each of the document images is the first page of the document, a method of using a page number of each of the document images, a method of using a margin area and a background color of each of the document images, and a method of using a title included in each of the document images; 2. The image reading device of claim 1, wherein the control unit causes the page number recognition unit to determine whether each of the document images is the first page of the document when the input accepting unit (i) accepts a selection of a method that uses the page numbers of each of the document images, (ii) causes the layout recognition unit to determine whether each of the document images is the first page of the document when the input accepting unit (ii) accepts a selection of a method that uses the margin areas and background color of each of the document images, and (iii) causes the title recognition unit to determine whether each of the document images is the first page of the document when the input accepting unit (iii) accepts a selection of a method that uses a title included in each of the document images.

5. the page number recognition unit recognizes a page number of each of the document images to determine whether or not each of the document images is a first page of a document; 5. The image reading device according to claim 1, wherein the dividing section divides the original image into documents of the same type based on the first page of the document determined by the page number recognition section.

6. the layout recognition unit recognizes a margin width and a background color from each of the document images to determine whether each of the document images is a first page of a document; 5. The image reading device according to claim 1, wherein the dividing section divides the original image into documents of the same type based on the first page of the document determined by the layout recognition section.

7. the title recognition unit extracts a character area of ​​a title from each of the document images to determine whether each of the document images is a first page of a document; 5. The image reading device according to claim 1, wherein the dividing section divides the original image into documents of the same type based on the first page of the document determined by the title recognition section.

Images