Apparatus, data structure and computer-implemented method for structuring digital data
The method addresses the challenge of structuring complex digital data by using a parser, OCR, and a foundation model to accurately extract and organize text from diverse formats, enhancing mechanical question-answering capabilities.
Patent Information
- Application Number
- JP2025126303
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-07-30
- Filing Date
- 2025-07-29
- Publication Date
- 2026-02-12
AI Technical Summary
Existing technologies struggle to efficiently structure digital data from complex formats like PDF files, leading to issues such as incorrect text ordering, mispositioning, and loss of essential information due to complex layouts and inconsistent storage methods.
A method involving a parser to read digital data, optical character recognition (OCR) on a pixel-dependent basis, and a foundation model to determine pixel arrangements, combined with a language model to infer structured digital data, iteratively correcting deviations and ensuring accurate text extraction.
Enables structured digital data representation, improving mechanical question-answering capabilities by accurately extracting and organizing text from diverse file types, including PDFs, while minimizing errors and hallucinations.
Smart Images
Figure 2026022637000001_ABST
Abstract
Description
[Technical Field]
[0001] Prior art The present invention relates to an apparatus, a data structure and a computer-implemented method for structuring digital data. [Background technology]
[0002] To be able to mechanically answer questions about the content of the digital data, it is advantageous for the digital data to exist in a structured state. Summary of the Invention [Means for solving the problem]
[0003] Disclosure of the Invention According to the computer-implemented method for structuring digital data as set forth in claim 1, it is assumed that digital data is provided, the digital data representing graphically displayable content, a description of an arrangement of pixels within the graphically displayable content representing the digital data is provided, and structured digital data representing the content, in particular the structure, the structured digital data, and the content filling the structure, are deduced depending on the digital data and the description of the arrangement of pixels. This means that the deduced digital data is structured depending on the description. This is advantageous for mechanically answering questions about the content.
[0004] For example, a file containing digital data may be provided, a parser configured to read the digital data from the file may be provided, and the digital data may be read from the file using the parser. This pre-processing allows different file types to be processed using corresponding parsers.
[0005] For example, pixels representing digital data are provided, and optical character recognition is provided that is configured to recognize the digital data in a pixel-dependent manner, where the digital data is recognized in a pixel-dependent manner using optical character recognition, which means that a digital image containing the content is provided for pre-processing, which allows different file types to be processed at the image level.
[0006] It may be assumed that a portion of the provided digital data represents a portion of the content, and a portion of the structured digital data represents a portion of the content, and the structured digital data is repeatedly estimated when it is recognized that the portions of the data deviate from each other. This means that the estimated structured digital data that deviates from the original digital data is replaced by other estimated structured digital data. This replaces hallucinations in the estimated structured digital data with the new estimated structured digital data.
[0007] It may be assumed that part of the provided digital data represents text, part of the structured digital data represents text, and the text in the structured digital data is deduced using a language model, depending on the text from the provided digital data and depending on the description, which means that part of the provided digital data is structured into text in order to improve mechanical responses to questions.
[0008] For example, a language model may be trained or instructed with a request to filter, classify, and / or process text from provided digital data in a manner dependent on the description of the request.
[0009] It may be assumed that a foundation model is provided that is configured to determine a pixel-dependent description of the array of pixels, and that the description of the array is determined pixel-dependently using the foundation model, which means that the foundation model provides a description that structures the data.
[0010] According to the data structure for structuring digital data, it is envisaged that the data structure comprises at least one data field for digital data, the digital data representing graphically displayable content, the data structure comprises at least one data field for a description of an arrangement of pixels representing the digital data within the graphically displayed content, and the data structure comprises at least one data field for structured digital data representing the content, deduced depending on the digital data and the description of the arrangement of pixels.
[0011] According to the apparatus for structuring digital data, it is envisioned that the apparatus includes at least one processor and at least one memory, the at least one memory including instructions executable by the at least one processor, and the apparatus performing the method when the instructions are executed by the at least one processor.
[0012] According to the computer program for structuring digital data, it is envisaged that the computer program comprises computer executable instructions which, when executed by a computer, cause the computer to carry out the method.
[0013] Further advantageous embodiments can be seen from the following description and drawings. [Brief explanation of the drawings]
[0014] [Figure 1] 1 is a schematic diagram of an apparatus for structuring digital data; [Figure 2]1 is a flowchart with steps for structuring digital data. [Figure 3] FIG. 1 is a diagram showing a data structure for structuring digital data. DETAILED DESCRIPTION OF THE INVENTION
[0015] In FIG. 1, an apparatus 100 for structuring digital data is shown schematically.
[0016] The device 100 includes at least one processor 102 and at least one memory 104 .
[0017] The at least one memory 104 includes instructions executable by the at least one processor 102 that, when executed by the at least one processor 102, cause the apparatus 100 to perform a method for structuring digital data.
[0018] FIG. 2 shows a flow chart with the steps of the method.
[0019] The method includes step 200 .
[0020] In step 200, a file containing digital data is provided.
[0021] The digital data from the file represents graphically displayable content, such as a document.
[0022] For example, a document may contain text that is displayed graphically in a left and right column per page when the document is displayed on a screen. The left column may contain, for example, German text. The right column may contain, for example, English text.
[0023] For example, a document may be divided into several different sections, and these sections may be separated into hierarchical levels. An example with three separate hierarchical levels is: 1XXX 1.1XXX 1.1.1XXX where XXX represents the text of each hierarchical level. The hierarchical levels are characterized by numbers separated from each other by dots.
[0024] The file contains, for example, text and instructions for graphical display.
[0025] An example of a file is a bilingual standard from the International Organization for Standardization (ISO).
[0026] The file is, for example, a file in PDF (Portable Document Format) format.
[0027] The method includes step 202 .
[0028] In step 202, digital data is provided.
[0029] For example, a parser configured to read digital data from a file is provided, for example, the digital data is read from the file using the parser.
[0030] The parser is configured to read text from a file, for example in PDF format, and the digital data read from the parser represents an unsorted list of sentences or words, for example from a file in PDF format.
[0031] The digital data provided by the parser is unstructured, for example in documents, for the following reasons:
[0032] Stored non-linearly: The displayed text may be stored in any order within the PDF format file. The displayed text can only be rendered in the correct order by linking to explicit position information from the PDF format file. Thus, the linear reading of individual elements of text by a parser will produce results that are clearly different from the text presented by a program for viewing PDF format files.
[0033] Complex layouts: Many PDF format files are not simply blocks of text, but often contain multiple columns, tables, images, and individual text boxes. This diversity makes general processing with a parser extremely difficult.
[0034] Inconsistent use of spaces: Within PDF, each word can be stored as an individual text element, which can result in no explicit spaces between words, making processing by a parser much more difficult.
[0035] For example, a parser is used to read the text level of a PDF file, which often results in incorrect text ordering and other mispositioning. Additionally, unnecessary text unrelated to the original text, such as copyright notices or page numbers, is often carried over.
[0036] Text in PDF files is often stored as an image rather than as a string of characters, which is the case, for example, when they are scanned, but can also have other causes, which prevents the stored text from being read.
[0037] If the file or text within the file is provided at least in part as a digital image, for example, pixels representing digital data of the digital image are provided, e.g., the digital data from the file is relied upon to determine the digital image, which includes graphically displayable content.
[0038] Optical character recognition is provided that is configured to recognize digital data on a pixel-dependent basis, e.g., with respect to text or other characters, e.g., digital data is recognized on a pixel-dependent basis, particularly from a digital image, using optical character recognition.
[0039] This means that PDF files are loaded as images and an attempt is made to recognize the text using optical character recognition on the document. Errors in recognition are unavoidable.
[0040] The method includes step 204 .
[0041] In step 204, a description of the arrangement of pixels representing digital data within the content to be displayed graphically is provided.
[0042] An exemplary description of a document is as follows:
[0043] The document contains text that is displayed graphically in a left and right column per page when the document is displayed on a screen: the left column contains text in German, the right column contains text in English.
[0044] The document is divided into several different sections, which are divided into three hierarchical levels, characterized by numbers separated from each other by dots.
[0045] It may be assumed that the description is specified by a user. It may be assumed that the description is automatically generated.
[0046] It may be assumed that a foundation model is provided that is configured to determine a description of the array of pixels in a pixel-dependent manner, e.g., the description of the array is determined in a pixel-dependent manner using the foundation model.
[0047] For example, a PDF file is loaded as an image by the underlying model and transferred to the latent space. In this case, essential information is lost because it is an approximation of the original document. Therefore, it cannot be guaranteed that each text element in the document can actually be reconstructed. In general, the likelihood that the underlying model can reproduce the exact text is rather low. However, the underlying model can describe the layout or language.
[0048] An example of the underlying model is described in "Visual Instruction Tuning (https: / / doi.org / 10.48550 / arXiv.2304.08485)."
[0049] For example, in the case of a document from a PDF file, a description of the layout, structure, or other characteristics of the PDF is generated. For example, the foundation model may be instructed to describe the layout of the pages of a document from a PDF file, whereby the foundation model recognizes and outputs, in this example, a left column and a right column, i.e., the characteristic "two columns."
[0050] In this case, if a specific characteristic is required, the user's request can be accommodated, for example, language.
[0051] It may be envisaged that the specified description is supplemented by the output of the underlying model.
[0052] The method includes step 206 .
[0053] In step 206, structured digital data representing the content is deduced depending on the digital data and the description of the arrangement of pixels.
[0054] It may be assumed that some of the digital data provided represents text.
[0055] This means that part of the structured digital data represents text.
[0056] It may be assumed that the text in the structured digital data is inferred using a language model, depending on the text from the provided digital data and depending on the description.
[0057] The language model is instructed, for example, with a request, to filter, classify, and / or process text from the provided digital data in accordance with the request and dependent on the description.
[0058] The language model may be trained or may be assumed to have been trained using appropriately selected training data to filter, classify, and / or process text from provided digital data in a description-dependent manner as required.
[0059] The specific output depends on the requirements. For example, for the ISO standard, two classified lists (one per language) can be output. An example of the structure of list 1 for German and list 2 for English is: List 1:[1:XXX,2:XXX,...] List 2: [1:XXX,2:XXX,...] is.
[0060] The sorted list shows an example of the structure of structured digital data, i.e., an example of a data structure, regardless of the content XXX. In this example, the structure includes a hierarchy, and the position of the content XXX in the hierarchy is characterized in this example by a number, which indicates the position in the data structure where the respective content XXX can be found in the data structure.
[0061] The method creates a structure of structured digital data. The method fills the structured digital data with content. The structured digital data is inferred in the method, which means that the structure and the content that fills this structure are inferred in the method.
[0062] Optionally, step 208 is performed.
[0063] According to step 208, it is assumed that deviations between the content represented by the estimated structured digital data and the content represented by the provided digital data are recognized.
[0064] This will be explained using the example of a portion of the provided digital data representing a part of the content and a portion of the structured digital data representing a part of the content.
[0065] The structured digital data is iteratively estimated if it is recognized that parts of the data deviate from one another. For this purpose, step 206 is repeated if parts of the data deviate from one another. Step 206 is repeated with corresponding suggestions, for example, about hallucination. The suggestions are, for example, specified as input to a language model.
[0066] When a language model is used, there is no guarantee that the output of the language model also reflects the original text, so for example, a method for recognizing hallucinations can be used to ensure that each sentence in the output of the language model also has a correspondence in the text of the document, i.e., in the text extracted from a file in PDF format. If an output is recognized that does not have a correspondence in the text, a corresponding indication can be issued that the output is hallucinated.
[0067] FIG. 3 shows a schematic diagram of a data structure 300 for structuring digital data.
[0068] The data structure 300 includes at least one data field 302 for provided digital data representing graphically displayable content.
[0069] The data structure 300 includes at least one data field 304 for a description of an arrangement of pixels representing digital data within the graphically displayed content.
[0070] The data structure 300 includes at least one data field 306 for structured digital data representing content that is deduced depending on the digital data and the description of the arrangement of pixels.
Claims
1. 1. A computer-implemented method for structuring digital data, comprising: the digital data is provided (202); the digital data represents graphically displayable content; a description of an arrangement of pixels representing said digital data within the graphically displayed content is provided (204); Depending on the digital data and the description of the arrangement of the pixels, structured digital data representing the content is deduced (206), in particular the structure, the structured digital data and the content filling the structure. A method characterized by:
2. a file containing said digital data is provided (200); a parser configured to read the digital data from the file is provided; The digital data is read from the file using the parser (202); The method of claim 1.
3. pixels representing said digital data are provided; an optical character recognition system configured to recognize the digital data in dependence on the pixels; The digital data is recognized in a pixel-dependent manner using the optical character recognition (202). The method of claim 1.
4. the portion of the provided digital data represents a portion of the content; a portion of the structured digital data representing a portion of the content; The structured digital data is iteratively estimated (206) when it is recognized that portions of the data deviate from one another (208).
4. The method according to any one of claims 1 to 3.
5. a portion of the provided digital data representing text; a portion of the structured digital data representing text; the text in the structured digital data is inferred (206) using a language model depending on the text from the provided digital data and depending on the description; 5. The method according to any one of claims 1 to 4.
6. The language model is trained or instructed with a request (206) to filter, classify, and / or process the text from the provided digital data according to the request and in dependence on the description. The method of claim 5.
7. a base model is provided that is configured to determine the description of the array of pixels in dependence on the pixels; The description of the array is determined (204) in a pixel-dependent manner using the underlying model.
7. The method according to any one of claims 1 to 6.
8. A data structure (300) for structuring digital data, comprising: The data structure (300) includes at least one data field (302) for digital data; the digital data represents graphically displayable content; the data structure (300) includes at least one data field (304) for a description of an arrangement of pixels representing the digital data within the graphically displayed content; the data structure (300) includes at least one data field (306) for structured digital data representing the content, deduced depending on the digital data and the description of the arrangement of the pixels; A data structure (300) comprising:
9. An apparatus (100) for structuring digital data, comprising: The device (100) comprises: at least one processor (102); At least one memory (104); Including, the at least one memory (104) includes instructions executable by the at least one processor (102); The instructions, when executed by the at least one processor (102), cause the device (100) to implement the method of any one of claims 1 to 7.
1. An apparatus (100) comprising:
10. 1. A computer program for structuring digital data, comprising: The computer program comprises computer-executable instructions, which, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 7. A computer program characterized by: