Information processing apparatus, control method of the same, and program

By analyzing and rendering graphic objects independently within a hierarchical document structure, the apparatus improves character recognition accuracy in outline text documents, addressing the accuracy issues of existing methods.

JP2024157964AInactive Publication Date: 2024-11-08CANON KK
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023072669
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-26
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing character recognition methods for full-page image and outline text documents suffer from decreased accuracy when characters have neutral colors or are on complex backgrounds, as they do not account for the hierarchical structure of drawing commands.

Method used

An information processing apparatus that analyzes drawing commands in a hierarchical structure, identifies graphic objects, renders them into bitmap images independent of the hierarchy, and performs OCR on these images to improve character recognition accuracy.

Benefits of technology

Enhances character recognition accuracy in documents with complex graphical elements by isolating and processing graphic objects separately, improving the ability to search, select, and read characters.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024157964000001_ABST
    Figure 2024157964000001_ABST
Patent Text Reader

Abstract

To solve the problem that in OCR processing when a character to be recognized is an intermediate color or when the character is placed on an image, character recognition accuracy for the character decreases.SOLUTION: A document expressed in a hierarchical structure including a plurality of drawing commands is input. The drawing commands in each hierarchy of the document are analyzed. It is determined whether the analyzed drawing command is a command for drawing a graphic object. When it is determined to be the graphic object, a rendering image of the graphic object is generated by rendering the graphic object. Character recognition processing is performed on the rendering image.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a control method thereof, and a program. [Background technology]

[0002] With the spread of online document management systems, more and more users are storing documents online. To facilitate browsing and searching of these documents, it is desirable to recognize the text contained in these documents and enable text search, text selection, copying, and reading aloud. For example, in documents created using Microsoft Word (registered trademark) or PowerPoint (registered trademark), text objects can be directly extracted from the document structure. This makes it easy to perform text search, text selection, copying, and reading aloud in these documents. However, documents in which the entire page has been converted into an image using a scanner (hereinafter referred to as a full-page image document) or documents in which the text has been converted into outlines (hereinafter referred to as an outline text document) do not contain information about text objects. Therefore, in order to perform text search, text selection, copying, and reading aloud in full-page image documents or outline text documents, text objects must first be identified.

[0003] Patent Document 1 describes a character recognition (OCR (Optical Character Recognition)) method for reading a document and recognizing characters in the document. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-102148 Summary of the Invention [Problem to be solved by the invention]

[0005] The above-mentioned method assumes a full-page image, and therefore even for documents with outline text, it is necessary to convert the document into a full-page image before performing OCR processing. However, OCR processing has the problem that the accuracy of character recognition decreases when the characters to be recognized are in neutral colors or on another image.

[0006] An object of the present invention is to solve at least one of the problems of the prior art.

[0007] An object of the present invention is to improve the accuracy of character recognition for documents expressed in a hierarchical structure including a plurality of drawing commands. [Means for solving the problem]

[0008] In order to achieve the above object, an information processing device according to one aspect of the present invention has the following configuration: an input means for inputting a document expressed in a hierarchical structure including a plurality of drawing commands; analysis means for analyzing drawing commands for each layer of the document; a determination means for determining whether the drawing command analyzed by the analysis means is a command for drawing a graphics object; a generating means for generating a rendering image of the graphics object by rendering the graphics object when the determining means determines that the graphics object is a graphics object; and character recognition means for performing character recognition processing on the rendering image. [Effects of the Invention]

[0009] According to the present invention, it is possible to improve the accuracy of character recognition for a document expressed in a hierarchical structure including a plurality of drawing commands.

[0010] Other features and advantages of the present invention will become apparent from the following description taken in conjunction with the accompanying drawings, in which the same or similar elements are designated by the same reference numerals. [Brief explanation of the drawings]

[0011] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention. [Figure 1] FIG. 1 is a block diagram showing an example of a hardware configuration of an information processing apparatus according to an embodiment of the present invention. [Figure 2] FIG. 2 is a functional block diagram showing an example of a software configuration of the information processing apparatus. [Figure 3A] FIG. 10 is a diagram showing an example of a command for page information in an outline character document. [Figure 3B] A plan view of the image corresponding to the command. [Figure 3C] A diagram showing a hierarchical structure of images corresponding to commands. [Figure 4] 10 is a flowchart illustrating character recognition of a page of an outline character document by an information processing apparatus according to an embodiment. [Figure 5] FIG. 3B is a diagram showing an example of a rendered image in which the graphic objects of the page information in FIG. 3A are rendered in a state independent of the hierarchical structure. [Figure 6] FIG. 2 is a diagram showing an example of a document with an OCR result according to the embodiment. [Figure 7] An example of a full-page image with the red text "1234" on an orange background. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0013] In this embodiment, an information processing device is used as an example to perform OCR processing on an outline character document expressed in a hierarchical structure containing multiple drawing commands, by imaging graphic objects that have been separated from the hierarchical structure.

[0014] FIG. 1 is a block diagram showing an example of a hardware configuration of an information processing device 100 according to an embodiment of the present invention.

[0015] The information processing device 100 includes a CPU 101, a RAM 102, a ROM 103, a network I / F (interface) 104, and a storage device 105, all of which are connected via a system bus 106. The CPU 101 is the central control unit of the information processing device 100. The RAM 102 is a storage device accessible by the CPU 101 and, in this embodiment, is used as a work memory for the CPU 101 to operate. The ROM 103 stores programs, and the CPU 101 loads and executes the programs on the RAM 102, thereby operating each software module shown in FIG. 2 (described later). The network I / F 104 is an interface connected to an external device 108 (such as a server or other device) via a network 107 and is used for inputting and outputting information. The storage device 105 is an auxiliary storage device such as an HDD or SSD, and is used as a work area for the CPU 101 and for storing data such as documents. In this embodiment, a document is acquired from the external device 108 via the network I / F 104, or from the storage device 105.

[0016] Fig. 2 is a functional block diagram showing an example of the software configuration of the information processing device 100. The information processing device 100 has software modules 201 to 206 shown in Fig. 2. As described above, these software modules operate when the CPU 101 executes a program loaded from the ROM 103 onto the RAM 102.

[0017] A document input by a document input unit 201 is analyzed by a document analysis unit 202 to obtain graphic objects from drawing commands in the document. A rendering unit 203 performs rendering processing to develop the obtained graphic objects into bitmap images with a specified resolution. An OCR unit 204 performs OCR processing on the image obtained by rendering. An OCR result combination unit 205 combines the character code and character position information obtained by the OCR unit 204. A document generation unit 206 adds the combined character code and character position information to the original document to generate a document with the OCR results.

[0018] 3A to 3C are diagrams illustrating the structure of an outline text document according to an embodiment. Here, simple XPS data is used as an example, but the data format handled may be other data formats such as PDF data instead of XPS data.

[0019] The document's page information includes commands related to the page width and height, as well as commands related to drawing. Each drawing command exists independently and is drawn from the back to the front in the order in which it is loaded. Therefore, if coordinates overlap, the drawing command loaded last is drawn in the foreground, creating a hierarchical structure. Furthermore, since outline text documents draw character commands as graphic objects specified by points and lines, they do not contain information about text objects such as fonts or character codes.

[0020] FIG. 3A is a diagram showing an example of a command for page information 300 in an outline text document. FIG. 3B is a diagram showing a planar view of the image corresponding to the command, and FIG. 3C is a diagram showing a hierarchical structure of the image corresponding to the command. Here, the drawing commands are arranged from the backmost to the foremost. Command 301, the "FixedPage" command, is a command related to the width and height of the page, with "Width="793.76"" indicating the page width and "Height="1122.56"" indicating the page height. The "Path Data" in commands 302 to 306 indicates drawing commands for graphic objects.

[0021] "F" in "Path Data" is the fill rule, F0 is EvenOdd, F1 is NonZero, "M x,y" is the starting point (x,y), "L x1,y1(,x2,y2,...)" is the sequence of points connecting the previous point with a straight line, "C x1,y1(,x2,y2,...)" is the sequence of points connecting the previous point with a Bezier curve, and "z" connects the previous point to the starting point. "RenderTransform" indicates the affine transformation matrix, "Fill" is the color to fill the shape with, and "#aarrggbb" indicates R=0xrr, G=0xgg, B=0xbb, α=0xaa.

[0022] Command 302, "Path Data," is the drawing command for the first graphic object loaded, and draws an orange rectangle with R=0xFF, G=0xFC, B=0xB0, and α=0xFF at the back (302 in Figures 3B and 3C).

[0023] The "Path Data" of command 303 is a drawing command for a graphic object, and draws the character "1" as a graphic in red with R=0xEE, G=0x1C, B=0x23, and α=0xFF on the rectangular object of command 302 (303 in Figures 3B and 3C). Similarly, the "Path Data" of commands 304 to 306 are drawing commands for graphic objects, and draw the characters "2," "3," and "4" as graphics on the rectangular object of command 302 (304, 305, and 306 in Figures 3B and 3C).

[0024] FIG. 4 is a flowchart illustrating character recognition of a page of an outline text document by the information processing device 100 according to the embodiment. This process is executed by the document input unit 201 for each page of a document acquired from the external device 108 via the network I / F 104 or acquired from the storage device 105. In the description of FIG. 4, the page information 300 of the outline text document shown in FIG. 3A will be used. The document here refers to a document expressed in a hierarchical structure including a plurality of drawing commands. The process illustrated in the flowchart of FIG. 4 is realized by the CPU 101 executing a program loaded in the RAM 102, as described above.

[0025] When functioning as the document analysis unit 202, the CPU 101 sequentially analyzes the drawing commands from the backmost to the frontmost in a document expressed in a hierarchical structure, starting from the backmost. In S401, the CPU 101 functions as the document analysis unit 202 and determines whether there are any unprocessed commands to be read into the page information. If it is determined that there are any unprocessed commands, the process proceeds to S402, and if it is determined that there are no unprocessed commands, the process proceeds to S408. In S402, the CPU 101 functions as the document analysis unit 202 and determines whether the read command at a certain layer is a drawing command and a graphics object. If it is determined that it is a graphics object, the process proceeds to S403, and if it is determined that it is some other command, the process proceeds to S407.

[0026] In S403, the CPU 101 functions as the rendering unit 203 and performs a so-called rendering process, which develops only the graphic objects of the acquired hierarchy into a bitmap image with a specified resolution. In the page information 300 of Fig. 3A, commands 302 to 306 correspond to the graphic objects. Then, each graphic object is rendered independently from the hierarchical structure, and graphic object images 501 to 505 that do not include back faces and are independent from the hierarchical structure are generated, as shown in Fig. 5.

[0027] FIG. 5 is a diagram showing an example of a rendered image obtained by rendering the graphic objects of the page information 300 in FIG. 3A in a state independent of the hierarchical structure.

[0028] Next, the process proceeds to S404, where the CPU 101 functions as the OCR unit 204 and performs OCR processing on the rendering images 501-505 of each graphic object. Next, the process proceeds to S405, where the CPU 101 functions as the OCR unit 204 and determines whether any of the recognized characters match the character code, i.e., whether the characters were recognized. If the characters were recognized, the process proceeds to S406; if the characters were not recognized, the process proceeds to S407. In S406, the CPU 101 functions as the OCR unit 204 and saves the character codes and character position information of the recognized characters obtained as the OCR results in the storage device 105. The specific means of OCR processing are the same as those of the prior art, so a description thereof will be omitted. Then, the process proceeds to S407, where the CPU 101 functions as the document analysis unit 202 and reads the next command. In this manner, the processes of S402-S407 are repeated until all commands in the page information have been processed.

[0029] 5, for example, there is no character that matches the character code for image 501, but there are characters that match the character codes for images 502 to 505. Therefore, in S404, the character codes corresponding to the characters "1," "2," "3," and "4" and the character position information corresponding to each character code are acquired, and in S406, the character codes and character position information of the characters that were recognized are associated and saved.

[0030] Once all commands have been processed in S401, the process proceeds to S408, where the CPU 101 functions as the OCR result combining unit 205 and combines the character codes and character position information within the page obtained by the OCR unit 204. The process then proceeds to S409, where the CPU 101 functions as the document generating unit 206 and adds the character codes and character position information within the combined page to the original document to generate a page with the OCR results.

[0031] By performing these processes on all pages, a document with OCR results is generated.

[0032] Fig. 6 is a diagram showing an example of a document with an OCR result according to the embodiment. In Fig. 6, parts common to those in Fig. 3 are given the same reference numerals, and descriptions thereof will be omitted.

[0033] This document 600 is a document in which character codes and character position information within the combined pages have been added to the original document 300 in Fig. 3A, and commands 301 to 306 of the original document 300 remain the same. In Fig. 6, command 601 indicating the character code and character position information has been added after command 306.

[0034] "Glyphs" in command 601 indicates the drawing command for the text object. "FontUri" in command 601 is the storage location of the referenced font, and "FontRenderingEmSize" is the font size. "StyleSimulations" indicates information about glyphs such as bold and italic. "OriginX" is the X coordinate of the starting point, "OriginY" is the Y coordinate of the starting point, and "Indices" indicates information specifying optional parameters that are not required, such as the index of the actual font data corresponding to the character code. "UnicodeString" indicates the text to be drawn.

[0035] Command 601 "Glyphs" draws the fully transparent string "1234" so that it overlaps the characters drawn as shapes by commands 303 to 306 in the original document 300. As a result, although only a fully transparent string of characters has been added to the appearance, the document is in a state where character information has been added, making it possible to search for characters, select and copy characters, and read them aloud.

[0036] As described above, according to the embodiment, when an outline character document is subjected to OCR, it is not converted into a full-page image and then OCR is performed, but graphic objects independent of the hierarchical structure are imaged and then OCR is performed.

[0037] In the example of page information 300 containing an outline character document in FIG. 3A, if OCR is to be performed on the full-page image, it is necessary to perform OCR on the image shown in FIG. 7. In the example of FIG. 7, the red characters "1234" are drawn on an orange background. In this case, character recognition is required by determining a threshold value that effectively separates the orange rectangular image from the red characters. In such cases, character recognition becomes more difficult when the characters are on a more complex image.

[0038] In contrast, in this embodiment, an independent graphic object is extracted from the hierarchical structure, and then imaged and character-recognized, which allows character recognition processing to be performed without being hindered by other objects, thereby improving the accuracy of character recognition for outline character documents.

[0039] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0040] The present specification and drawings disclose the following information processing device, its control method, and program.

[0041] [Item 1] an input means for inputting a document expressed in a hierarchical structure including a plurality of drawing commands; analysis means for analyzing drawing commands for each layer of the document; a determination means for determining whether the drawing command analyzed by the analysis means is a command for drawing a graphics object; a generating means for generating a rendering image of the graphics object by rendering the graphics object when the determining means determines that the graphics object is a graphics object; character recognition means for performing character recognition processing on the rendering image; An information processing device comprising:

[0042] [Item 2] 2. The information processing device according to item 1, wherein the analysis means sequentially analyzes drawing commands from the backmost to the foremost of the document, starting from the backmost.

[0043] [Item 3] 3. The information processing device according to item 1 or 2, wherein the drawing commands are arranged in the document in order from the backmost drawing command to the foremost drawing command.

[0044] [Item 4] The information processing device according to any one of items 1 to 3, characterized in that the determination means determines that the drawing command is a command to draw a graphics object when the drawing command includes a command indicating an image specified by points and lines.

[0045] [Item 5] 5. The information processing device according to any one of items 1 to 4, wherein the rendering image is a bitmap image with a predetermined resolution.

[0046] [Item 6] 6. The information processing device according to any one of items 1 to 5, wherein the character recognition means acquires a character code of a recognized character and position information indicating a position of the character.

[0047] [Item 7] 7. The information processing apparatus according to item 6, further comprising a means for generating a document by adding the character code and the position information acquired by the character recognition means to the document.

[0048] [Item 8] 8. The information processing device according to any one of items 1 to 7, wherein the document is an outline character document in which characters are converted into outlines.

[0049] [Item 9] A control method for controlling an information processing device, comprising: an input step of inputting a document expressed in a hierarchical structure including a plurality of drawing commands; an analysis step of analyzing drawing commands for each layer of the document; a determining step of determining whether the drawing command analyzed in the analyzing step is a command for drawing a graphics object; a generating step of generating a rendering image of the graphics object by rendering the graphics object when the determining step determines that the graphics object is a graphics object; a character recognition step of performing character recognition processing on the rendering image; A control method comprising:

[0050] [Item 10] A program for causing a computer to function as each of the means of the information processing device described in any one of items 1 to 8.

[0051] The present invention is not limited to the above-described embodiments, and various modifications and variations can be made without departing from the spirit and scope of the present invention. Therefore, the following claims are appended to apprise the public of the scope of the present invention. [Explanation of symbols]

[0052] 202... document analysis unit, 203... rendering unit, 204... OCR unit, 205... OCR result combination unit, 206... document generation unit

Claims

1. an input means for inputting a document expressed in a hierarchical structure including a plurality of drawing commands; an analysis means for analyzing drawing commands of each layer of the document; a determination means for determining whether the drawing command analyzed by the analysis means is a command for drawing a graphics object; a generating means for generating a rendering image of the graphical object by rendering the graphical object when the determining means determines that the graphical object is a graphical object; character recognition means for performing character recognition processing on the rendering image; 13. An information processing device comprising:

2. 2. The information processing apparatus according to claim 1, wherein said analyzing means analyzes drawing commands from the backmost to the frontmost of said document in sequence from the backmost to the frontmost of said document.

3. 2. The information processing apparatus according to claim 1, wherein the drawing commands are arranged in the document in order from the backmost drawing command to the foremost drawing command.

4. 2. The information processing apparatus according to claim 1, wherein said determining means determines that the drawing command is a command for drawing a graphics object when the drawing command includes a command for showing an image specified by points and lines.

5. 2. The information processing apparatus according to claim 1, wherein the rendering image is a bitmap image having a predetermined resolution.

6. 2. The information processing apparatus according to claim 1, wherein said character recognition means acquires a character code of a recognized character and position information indicating a position of the recognized character.

7. 7. The information processing apparatus according to claim 6, further comprising: a means for generating a document by adding the character code and the position information acquired by the character recognition means to the document.

8. 2. The information processing apparatus according to claim 1, wherein the document is an outline character document in which characters are converted into outlines.

9. A control method for controlling an information processing device, comprising: An input step of inputting a document expressed in a hierarchical structure including a plurality of drawing commands; an analysis step of analyzing drawing commands of each layer of the document; a determination step of determining whether the drawing command analyzed in the analysis step is a command for drawing a graphics object; a generating step of generating a rendering image of the graphical object by rendering the graphical object when the determining step determines that the graphical object is a graphical object; a character recognition step of performing character recognition processing on the rendering image; A control method comprising the steps of:

10. A program for causing a computer to execute each step of the control method according to claim 9.

Citation Information

Patent Citations

  • Image processing device, image processing method, and program

    JP2020102148A