Method for processing document and system therefor

Abstract

A document processing method and a system are provided. A document processing method according to some embodiments may include the steps of converting a target document into a non-editable document; extracting one or more document elements constituting the target document based on the non-editable document; inputting a parsing prompt based on one or more document elements into a generative language model to obtain processed text corresponding to the target document; and chunking the processed text to divide it into a plurality of passages. According to this method, a high-quality passage database capable of contributing to the performance improvement of a knowledge service can be effectively constructed.

Description

Technology Field

[0001] The present disclosure relates to a technology for processing documents for a Retrieval-Augmented Generation (RAG) based knowledge service. Background Technology

[0002] Recently, in knowledge services utilizing generative language models, the Retrieval-Augmented Generation (RAG) method, which generates answers by referencing passages retrieved from external documents, is widely used. The performance of this RAG method is largely determined by the quality of the passages themselves and the search accuracy of related passages. In other words, the more faithfully passages reflect the structure, semantics, and context of documents, and the more accurately passages related to the user's query are retrieved, the higher the accuracy of the answers provided by the knowledge service becomes. Therefore, the sophisticated processing and chunking of documents are crucial during the process of building a passage database.

[0003] However, conventional document processing and chunking methods fail to adequately reflect these requirements. Conventional methods remain at the level of simply converting non-text elements within a document (e.g., images, charts, tables, diagrams, etc.) into text, and employ a mechanical chunking method that divides the text into fixed-length segments. As a result, passages are generated where context is disrupted or the structural meaning of the document is not reflected, and it is difficult to secure sufficient metadata for passage searching (information that can improve search performance, such as headings and summaries).

[0004] Consequently, conventional methods have a clear limitation in that it is difficult to guarantee the answer accuracy required for RAG-based knowledge services. Prior art literature

[0005] Korean Registered Patent No. 10-2321707 (Published Nov. 04, 2021) The problem to be solved

[0006] Various technical problems to be solved through some embodiments of the present disclosure relate to document processing methods and systems.

[0007] Specifically, the technical problem to be solved through some embodiments of the present disclosure is to provide a document processing method and a system capable of constructing a high-quality passage database for a Retrieval-Augmented Generation (RAG)-based knowledge service.

[0008] In addition, another technical problem to be solved through some embodiments of the present disclosure is to provide a method and a system capable of accurately providing source information (i.e., answer source) of a passage referenced in a RAG-based knowledge service.

[0009] In addition, another technical problem to be solved through some embodiments of the present disclosure is to provide a method and a system capable of providing rich answers containing non-text document elements (e.g., images, charts, etc.) in a RAG-based knowledge service.

[0010] In addition, another technical problem to be solved through some embodiments of the present disclosure is to provide a document processing method and a system capable of improving the search accuracy of associated passages.

[0011] The technical problems of the present disclosure are not limited to those mentioned above, and other unmentioned technical problems will be clearly understood by a person skilled in the art of the present disclosure from the description below. means of solving the problem

[0012] A document processing method according to some embodiments of the present disclosure for solving the aforementioned technical problem may include, in a method performed by at least one processor, the step of converting a target document into a non-editable document; the step of extracting one or more document elements constituting the target document based on the non-editable document; the step of inputting a parsing prompt based on the one or more document elements into a generative language model to obtain processed text corresponding to the target document; and the step of chunking the processed text to divide it into a plurality of passages.

[0013] In some embodiments, the non-edited document may be a document in PDF (Portable Document Format) format.

[0014] In some embodiments, the target document may be a document in a file format configured so that at least one non-text document element is displayed through rendering.

[0015] In some embodiments, the step of extracting one or more document elements may include: extracting one or more non-text objects from the non-edited document and placing them in a new non-edited document; and applying an outline detection technique or an object detection technique to an image of the new non-edited document to extract non-text document elements.

[0016] In some embodiments, the step of extracting one or more document elements may include: extracting location information and size information of a specific non-text object from internal data of the target document; and extracting an area corresponding to the specific non-text object from an image of the non-edited document using the location information and the size information.

[0017] In some embodiments, the step of extracting one or more document elements may include the step of extracting text document elements from the target document; and the step of extracting non-text document elements from the non-edited document.

[0018] In some embodiments, the step of obtaining the processed text may include: a step of analyzing the layout of the non-edited document using at least one of the generative language model and another generative language model; and a step of generating the processed text by inputting the parsing prompt, which is further based on the analysis result of the layout, into the generative language model.

[0019] In some embodiments, the step of analyzing the layout of the non-edited document may include the step of performing a layout analysis of the non-edited document by inputting a prompt containing information of one or more document elements and an image of the non-edited document into the at least one model.

[0020] In some embodiments, the non-edited document includes a first page and a second page which is the next page of the first page, and the step of analyzing the layout of the non-edited document may include: a step of obtaining a first layout analysis result for the first page by inputting a first prompt containing document element information of the first page and an image of the first page into the at least one model; a step of configuring a second prompt to include at least a portion of the first layout analysis result and document element information of the second page; and a step of obtaining a second layout analysis result for the second page by inputting the second prompt and an image of the second page into the at least one model.

[0021] In some embodiments, the analysis result of the layout includes information regarding a first document element and a second document element that reflects the layout information, and the step of generating the processed text may include: obtaining a first element parsing prompt corresponding to the first document element and a second element parsing prompt corresponding to the second document element from a pre-prepared set of prompts; configuring the parsing prompt by combining the first element parsing prompt and the second element parsing prompt; and inputting the parsing prompt into the generative language model.

[0022] In some embodiments, the non-edited document includes a first page and a second page which is the next page of the first page, and the analysis result of the layout includes a first layout analysis result for the first page and a second layout analysis result for the second page, and the step of generating the processed text may include: a step of obtaining a first parsing result for the first page by inputting a first parsing prompt including the first layout analysis result and an image of the first page into the generative language model; a step of configuring a second parsing prompt such that at least a portion of the first parsing result and the first layout analysis result is included; and a step of obtaining a second parsing result for the second page by inputting the second parsing prompt and an image of the second page into the generative language model.

[0023] In some embodiments, the step of generating the processed text may include: obtaining an initial processed text output by the generative language model; converting the initial processed text into tag-formatted text; and, if the initial processed text is successfully converted into tag-formatted text, converting the tag-formatted text into the processed text in markdown format.

[0024] In some embodiments, certain passages among the passages are stored in a passage database along with metadata, the metadata includes a Uniform Resource Identifier (URI) of the non-edited document, and when an answer to a user query is provided based on the specific passage, source information of the answer may be provided using the URI.

[0025] In some embodiments, the specific passage is associated with a specific page of the non-edited document, and the URI may include a fragment specifying the specific page.

[0026] In some embodiments, the one or more document elements include non-text document elements, and an image of the non-text document element is stored at a specific location, and when an answer associated with the non-text document element is provided in response to a user's query: a Uniform Resource Identifier (URI) pointing to the specific location is included in the answer, and during the process of providing the answer, an image of the non-text document element may be displayed on the user's terminal via the URI.

[0027] A document processing system according to some embodiments of the present disclosure for solving the technical problem described above comprises: one or more processors; and a memory for storing a computer program executed by the one or more processors, wherein the computer program may include instructions for: converting a target document into a non-editable document; extracting one or more document elements constituting the target document based on the non-editable document; inputting a parsing prompt based on the one or more document elements into a generative language model to obtain processed text corresponding to the target document; and chunking the processed text to divide it into a plurality of passages.

[0028] A computer program according to some embodiments of the present disclosure for solving the technical problem described above may be stored in a non-transitory computer-readable recording medium to execute the steps of: converting a target document into a non-editable document; extracting one or more document elements constituting the target document based on the non-editable document; inputting a parsing prompt based on the one or more document elements into a generative language model to obtain processed text corresponding to the target document; and chunking the processed text into a plurality of passages. Effects of the invention

[0029] According to some embodiments of the present disclosure, a target document is converted into a non-editable document, and non-text document elements (e.g., images, charts, tables, diagrams, formulas, flowcharts, etc.) can be extracted from the said non-editable document. In this case, various non-text document elements that are difficult to extract from the original target document can be extracted accurately and easily.

[0030] In addition, the target document can be converted into a non-edited document in the highly compatible PDF (Portable Document Format) format, and a URI (Uniform Resource Identifier) ​​pointing to its storage location can be generated. Furthermore, if a passage belonging to the target document is utilized to generate an answer in a knowledge service, the source of the answer can be accurately presented using the corresponding URI, and the source document can be easily viewed without the need for a dedicated document program.

[0031] In addition, since the URI associated with each passage includes page information of the document to which the passage belongs, detailed page-level source information can be provided to users of the knowledge service. Accordingly, users can directly access the source page of the answer through the corresponding URI.

[0032] Additionally, an image of a non-text document element may be stored at a specific location, and a URI pointing to that location may be generated. Then, by providing the corresponding URI along with the answer from the knowledge service, the non-text document element can be displayed on the user's terminal, and as a result, a rich answer can be provided to the user.

[0033] In addition, by providing a prompt containing document element information and a non-edited document image together to a generative language model, the layout of the non-edited document can be accurately analyzed.

[0034] In addition, based on the layout analysis results, each document element present in the non-edited document is identified, and a parsing prompt can be generated by dynamically combining the corresponding element parsing prompts. In this case, the degradation of parsing accuracy and hallucination issues caused by the inclusion of parsing instructions for document elements that do not exist in the document in the prompt can be effectively prevented.

[0035] In addition, by providing a non-edited document image and a parsing prompt including the layout analysis results of the non-edited document together to a generative language model, parsing of the non-edited document (or target document) can be performed with precision.

[0036] In addition, by controlling the generative language model to separate layout analysis and document parsing tasks so that they are performed independently, the accuracy and processing stability of each task can be significantly improved.

[0037] In addition, by reflecting at least some of the layout analysis results of the previous page in the layout-related prompts of the next page, page-level layout analysis of a non-edited document (or target document) can be performed more accurately.

[0038] In addition, by reflecting at least some of the parsing results from the previous page in the parsing prompt of the next page, page-by-page parsing operations for a non-edited document (or target document) can be performed more precisely.

[0039] In addition, the parsing result output by the generative language model (i.e., the initial processed text) is converted into tag-formatted text, and if this conversion is successful, the tag-formatted text can be converted into the final processed text in Markdown format. In this case, syntax error issues that may occur in the Markdown format can be prevented in advance. For example, syntax error issues that frequently occur when the generative language model directly outputs processed text in Markdown format can be effectively resolved.

[0040] In addition, during the process of converting the initial processed text into tag-formatted text, various data / information regarding non-text document elements may be augmented and included in passages and / or their metadata. In this case, as non-text document elements are reflected in the passage search process, the search accuracy for related passages can be significantly improved.

[0041] In addition, processed text corresponding to the target document is generated to include structural information of the document, and the processed text can be chunked based on this structural information. In this case, passages with minimized contextual disconnection can be accurately generated, and as a result, a high-quality passage database capable of improving the accuracy of answers in knowledge services can be effectively constructed.

[0042] The effects according to the technical concept of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below. Brief explanation of the drawing

[0043] FIG. 1 is an exemplary drawing for explaining the operation of a document processing system according to some embodiments of the present disclosure at the system level. FIG. 2 is an exemplary drawing for further explaining the operation of a document processing system according to some embodiments of the present disclosure. FIG. 3 is an exemplary flowchart illustrating a document processing method according to some embodiments of the present disclosure. FIG. 4 is an exemplary drawing for illustrating a method for extracting document elements according to some embodiments of the present disclosure. Figure 5 is an exemplary drawing for explaining the detailed process of the layout analysis step illustrated in Figure 3. FIG. 6 illustrates an output format of a layout analysis result that may be referenced in some embodiments of the present disclosure. FIG. 7 is an exemplary drawing for illustrating a case where layout analysis is performed on a page-by-page basis according to some embodiments of the present disclosure. FIGS. 8 and 9 are exemplary drawings for explaining the detailed process of the processed text generation step illustrated in FIG. 3. FIG. 10 is an exemplary drawing illustrating a case where parsing is performed on a page-by-page basis according to some embodiments of the present disclosure. FIG. 11 is an exemplary flowchart illustrating a post-processing method according to some embodiments of the present disclosure. FIG. 12 is an exemplary drawing for further explaining a post-processing method according to some embodiments of the present disclosure. FIG. 13 is a drawing illustrating an example related to a method for data enrichment of non-text document elements according to some embodiments of the present disclosure. FIGS. 14a and FIGS. 14b are drawings illustrating other examples related to a method for data enrichment of non-text document elements according to some embodiments of the present disclosure. FIGS. 15a and FIGS. 15b are drawings illustrating another example related to a method for data enrichment of non-text document elements according to some embodiments of the present disclosure. FIGS. 16a and FIGS. 16b are drawings illustrating another example related to a method for data enrichment of non-text document elements according to some embodiments of the present disclosure. FIG. 17 illustrates a passage and metadata that may be referenced in some embodiments of the present disclosure. FIG. 18 is an exemplary drawing for explaining a method of providing knowledge services according to some embodiments of the present disclosure. FIG. 19 illustrates an exemplary computing device capable of implementing a document processing system according to some embodiments of the present disclosure. Specific details for implementing the invention

[0044] Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the attached drawings. The advantages and features of the present disclosure and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the attached drawings. However, the technical concept of the present disclosure is not limited to the following embodiments but can be implemented in various different forms. The following embodiments are provided merely to complete the technical concept of the present disclosure and to fully inform those skilled in the art of the scope of the present disclosure, and the technical concept of the present disclosure is defined only by the scope of the claims.

[0045] In describing the various embodiments of the present disclosure, if it is determined that a detailed description of related known configurations or functions could obscure the essence of the present disclosure, such detailed description is omitted.

[0046] Unless otherwise defined, terms used in the following embodiments (including technical and scientific terms) may be used in a meaning commonly understood by those skilled in the art to which this disclosure pertains, but this may vary depending on the intent of those skilled in the art, case law, the emergence of new technology, etc. The terms used in this disclosure are for describing the embodiments and are not intended to limit the scope of this disclosure.

[0047] In the following embodiments, singular expressions include plural concepts unless the context clearly specifies them as singular. Additionally, plural expressions include singular concepts unless the context clearly specifies them as plural.

[0048] In addition, terms such as first, second, A, B, (a), (b), etc. used in the following embodiments are used merely to distinguish one component from another, and the essence, order, or sequence of the said component is not limited by such terms.

[0049] In the following embodiments, components described using terms such as ~part or unit, module, block, ~or, ~er, etc., and functional blocks illustrated in the drawings may be implemented in the form of software, hardware, or a combination thereof. Software may include, for example, machine code, firmware, embedded code (or software), application software, or a combination thereof. Additionally, hardware may include, for example, electrical circuits, electronic circuits, processors, computers, integrated circuits, integrated circuit cores, passive components, or a combination thereof. As more specific examples, such as ~part, module, etc., may include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables.

[0050] Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0051] FIG. 1 is an exemplary drawing for explaining the operation of a document processing system (10) according to some embodiments of the present disclosure at the system level.

[0052] As illustrated in FIG. 1, the document processing system (10) is a computing system (or device) that precisely processes a target document (12) to build a high-quality passage DB (11). For example, the document processing system (10) can convert the target document (12) into processed text (13) containing document structure information (e.g., document hierarchy information), and divide (or generate) multiple passages by chunking the processed text (13) considering the maximum passage length and document structure. Then, the document processing system (10) can appropriately process each passage and store it in the passage DB (11, database). By repeating this process for various documents, the document processing system (10) can build a high-quality passage DB (11) that can be utilized in a RAG (Retrieval-Augmented Generation) based knowledge service.

[0053] FIG. 2 is an exemplary drawing for further explaining the operation of a document processing system (10) according to some embodiments of the present disclosure. Hereinafter, for convenience of explanation, the document processing system (10) will be abbreviated as 'system'.

[0054] As illustrated in FIG. 2, the system (10) can first convert the target document (22) into a non-editable document (23). At this time, the system (10) can extract one or more document elements (e.g., text, image, etc.) from the target document (22), which will be described later.

[0055] The target document (22) may be a document in an editable format (e.g., file format), such as PowerPoint (e.g., ppt, pptx file, etc.), Word (e.g., doc, docx file, etc.), or text format (e.g., txt file, etc.), but the scope of the present disclosure is not limited thereto. If the target document (22) is in a 'non-editable format', the document conversion process may be omitted.

[0056] Non-editable documents (23) may refer to documents that have restrictions on editing, such as PDF (Portable Document Format) and image formats (e.g., png, jpg files, etc.), such as general PDF documents, scanned PDF documents, document images, scanned document images, etc.

[0057] The reason for converting the target document (22) into a non-edited document (23) is that, generally, non-text document elements (e.g., charts, diagrams, etc.) other than images included in the target document (22) are not stored in an extractable form and are configured to be displayed through rendering by a dedicated document program. For example, PowerPoint or Word documents store data such as charts and diagrams in an internal format (or structure) defined internally, and are configured to be displayed by rendering by a dedicated document program. Therefore, it is difficult and inefficient to directly extract non-text document elements such as charts and diagrams from these documents.

[0058] Another reason is that non-edited documents (23), such as PDF format, have excellent compatibility and can be easily displayed (or viewed) in various programs (e.g., web browsers, etc.) without a dedicated document program. By utilizing these characteristics, if non-edited documents (23) are hosted on a server so that they can be accessed via a Uniform Resource Identifier (URI), the system (20) providing RAG-based knowledge services can present the source of the answer (e.g., the document to which the referenced passage belongs) in a form that allows the user to immediately verify (or view) it using the URI. For further details on this, refer to the explanation in FIG. 18. Additionally, by using the URI, non-edited documents (23) can be provided to the reviewer of the document processing process in a form that can be immediately viewed.

[0059] Next, the system (10) can generate processed text (24) corresponding to a non-edited document (23) using a generative language model (21). This will be explained in detail with reference to the drawings from Fig. 3 onwards.

[0060] A generative language model (21) may refer to a deep learning model equipped with natural language understanding and generation capabilities. Additionally, the scope of the generative language model (21) may include multi-modal deep learning models equipped with image understanding capabilities, such as a Vision-Language Model (VLM). Depending on the case, such a generative language model (21) may be named as a Large Language Model (LLM), Artificial Intelligence (Generative AI), generative model, language model, multimodal model, etc.

[0061] The processed text (24) may be, for example, text in Markdown format, but the scope of the present disclosure is not limited thereto. The Markdown format has the advantage of not only being able to clearly express structural information of a document, but also having high readability due to its simple syntax and excellent compatibility in various environments.

[0062] Next, the system (10) can chunk the processed text (24) to divide it into multiple passages (25). For example, the system (10) can chunk the processed text (24) based on the maximum passage length (e.g., maximum number of tokens in the passage) and document structure information (e.g., hierarchical structure, etc.) contained in the processed text (24) (e.g., chunking into variable lengths so that sections such as chapters, sections, etc. are not separated into different passages).

[0063] Next, the system (10) can map each of the multiple passages (25) to a passage embedding (e.g., embedding vector) and store them in the passage DB (11). The specific method of embedding each passage can be any method. For example, the system (10) can embedding each passage using a text embedding model.

[0064] The passage DB (11) can be understood as a knowledge base that supports vector search, and in addition to vector search, it may also support search based on various metadata (e.g., adjacent / associated passage search using document structure information such as headings, passage summary-based keyword search, text-based search for non-text document elements, etc.).

[0065] The system (10) can build a high-quality passage DB (11) by repeating the above-described process for other documents, and the passage DB (11) can be utilized by a system (20) that provides RAG-based knowledge services. For example, the knowledge service system (20) can search the passage DB (11) to obtain passages related to a user query, and input a prompt based on the user query and related passages into a generative language model to generate an accurate answer. The generated answer can be provided to the user.

[0066] This knowledge service system (20) may be named as a ‘chatbot service system’, ‘question answering system’, ‘RAG-based system’, ‘RAG service system’, etc. depending on the case, and the term system may be used interchangeably with terms such as ‘device’, ‘server’, ‘platform’, etc.

[0067] The knowledge service system (20) may be a separate system distinct from the system (10) described above, or it may be the same system.

[0068] The above-described system (10) may be implemented with at least one computing device. For example, all functions of the system (10) may be implemented in a single computing device, or a first function of the system (10) may be implemented in a first computing device and a second function may be implemented in a second computing device. Alternatively, a specific function of the system (10) may be implemented in multiple computing devices.

[0069] A computing device may include any device equipped with computing functions, and for an example of such a device, refer to FIG. 19. Since a computing device is a collection of various components (e.g., memory, processor, etc.) that interact, it may be referred to as a 'computing system' depending on the case. Of course, the term computing system may also encompass the concept of a collection of multiple computing devices that interact.

[0070] Up to now, the operation of a system (10) according to some embodiments of the present disclosure has been schematically described with reference to FIGS. 1 and 2. Hereinafter, various methods related to document processing will be described with reference to FIGS. 3 and subsequent drawings.

[0071] The methods described below may be performed by at least one processor (hereinafter abbreviated as 'processor') or implemented in said processor. For example, at least some of the steps / operations of the methods described below may be performed by a processor equipped with the system (10) described above.

[0072] For the sake of ease of understanding, the following explanation assumes that all steps / operations of the methods described below are performed by a processor of a specific computing system (or device). Therefore, if the subject of a specific step / operation is omitted, it can be understood that the corresponding step / operation is performed by that processor. However, in actual environments, some steps / operations of the methods described below may be performed on a different computing system.

[0073] FIG. 3 is an exemplary flowchart illustrating a document processing method according to some embodiments of the present disclosure. However, this is merely an exemplary embodiment for achieving the purpose of the present disclosure, and it is understood that some steps may be added or deleted as necessary.

[0074] As illustrated in FIG. 3, the document processing method according to the embodiments may begin at step S31, which involves obtaining a target document. The target document may be, for example, a document in an editable format, such as a document in a file format configured to display at least one non-text document element through rendering by a dedicated document program (e.g., PowerPoint, Word document, etc.). However, the scope of the present disclosure is not limited thereto. Nevertheless, for the sake of convenience of understanding, the following description will continue by assuming that the target document is an 'editable format document'. Additionally, the target document may be, for example, a document with page divisions and may consist of one or more pages, but the scope of the present disclosure is not limited thereto. If the target document consists of multiple pages, at least some steps (e.g., steps S33 to S36, etc.) may be performed on a page-by-page basis, which will be described later.

[0075] In step S32, the target document is converted into a non-edited document. For example, the system (10) can convert the target document into a PDF format (or image format, etc.).

[0076] In some embodiments, the unedited document is stored at a specific location (e.g., a server, etc.), and a URI pointing to that location may be generated (or assigned). That is, the unedited document may be configured to be accessible via a URI. This URI may be used to provide the unedited document to a reviewer during the document processing process or to present the source of the answer in a knowledge service, for which further reference is made to the description in FIG. 18.

[0077] In step S33, one or more document elements constituting the target document are extracted based on the non-edited document. For example, the system (10) can extract non-text document elements such as images, tables, charts, diagrams, formulas, flowcharts, etc. from the non-edited document. Document elements may be abbreviated as 'elements' depending on the case.

[0078] The specific method for extracting document elements may vary depending on the example.

[0079] In some embodiments, document elements can be extracted from a non-edited document using an intelligent document analysis service. For example, the system (10) can extract text document elements (or text) from a non-edited document using the Optical Character Recognition (OCR) function of the intelligent document analysis service, or extract a table from a non-edited document using the table extraction function. An intelligent document analysis service is a service that provides various document analysis and / or understanding functions based on Artificial Intelligence (AI), examples of which may include Azure Document Intelligence, but the scope of the present disclosure is not limited thereto. Commercially available intelligent document analysis services are known to perform OCR or table extraction functions with significantly high accuracy.

[0080] In some other embodiments, one or more non-text objects are extracted from a non-edited document and placed in a new non-edited document, and then non-text document elements can be extracted by applying an outline detection technique or an object detection technique to an image of the new non-edited document. Here, an object refers to a unit element such as text, image, or shape (e.g., line, curve, rectangle, circle, etc.) managed in the non-edited document, and may or may not correspond one-to-one with a document element. For example, as illustrated in FIG. 4, let us assume that the non-edited document (41) is a 'PDF format document'. In this case, the system (10) can extract only non-text objects (45 to 47) from the non-edited document (41), excluding text objects (42, 43), and place them in a new non-edited document (48, e.g., an empty document of the same size as the non-edited document (41)). This operation may be implemented by parsing (or extracting) information of non-text objects using a PDF library and then performing rendering (i.e., redrawing objects) based on the object information in a new non-edited document, but the scope of the present disclosure is not limited thereto. Next, the system (10) may convert the new non-edited document (48) into an image (49) and apply an outline detection technique and / or an object detection technique to the image (49) to extract non-text document elements. For example, the system (10) may use an outline detection technique to detect the area of ​​a non-text object (e.g., a rectangular area) and, if adjacent non-text objects exist, repeat the process of merging the object areas (e.g., a table is composed of multiple line objects). Next, the system (10) may extract non-text document elements in image format from the final object area. At this time, the system (10) may store the image of the non-text document element at a specific location and generate and manage a URI pointing to that location.This URI can be used when providing answers in a knowledge service, and further details regarding this are provided in the explanation of Fig. 18.

[0081] In the preceding embodiments, the system (10) can determine the type of non-text document element present in the final object area based on the number of line objects, the slope (angle), the number of parallel line objects, the outer pixel value of the final object area, etc. For example, if the number of line objects is greater than or equal to a threshold and the ratio of parallel line objects to the total number of line objects is greater than or equal to the threshold relative to the total, the system (10) can determine that the document element in the final object area is a table. Conversely, it can be determined to be an image.

[0082] In some other embodiments, location and size information of one or more non-text objects is extracted from the internal data of a target document, and using the extracted information, an area corresponding to one or more non-text objects in an image of a non-edited document can be extracted as a non-text document element. For example, let's assume the target document is a 'PowerPoint document'. In this case, the system (10) can obtain the internal data of the target document (e.g., XML data) using a presentation library and extract coordinate information indicating the size and location of non-text objects (e.g., drawing objects such as lines and shapes, images, etc.) from the data. Then, the system (10) can repeatedly merge areas of adjacent non-text objects based on the coordinate information, or merge areas of text objects (e.g., captions, etc.) adjacent to non-text objects as needed. Then, the system (10) can crop an area corresponding to the final object area in the image of the non-edited document to extract a non-text document element (e.g., chart, diagram, etc.) in image format. Similarly to the above, the extracted image can be stored at a specific location and assigned a URI pointing to that location.

[0083] In some other embodiments, text document elements may be extracted by analyzing a non-edited document. Any specific extraction method may be used.

[0084] In some other embodiments, document elements may be extracted based on various combinations of the embodiments described above.

[0085] Meanwhile, document elements such as text and images may be extracted directly from the target document. This is because direct extraction is easy when images are managed as separate resources within the document data (e.g., stored in an internally independent folder). For example, the system (10) can extract text document elements and image elements from the target document and extract non-text document elements other than images using a non-edited document. Alternatively, the system (10) can extract text document elements from the target document and extract non-text document elements using a non-edited document. However, the specific extraction method can be designed in various ways.

[0086] For example, the system (10) can extract text document elements (or text objects) using a library of the target document. For instance, if the target document is a 'PowerPoint document', the system (10) can extract internal XML data of each page (slide) using a presentation library and parse it to extract text document elements (or text objects).

[0087] As another example, the system (10) can extract non-text document elements (or non-text objects) using a library of the target document. For instance, if the target document is a 'PowerPoint document', the system (10) can extract internal XML data of each page (slide) using a presentation library and parse it to obtain table data.

[0088] As another example, the system (10) can extract headings using style information of the target document. For instance, if the target document is a 'Word document', the system (10) can parse a style-related XML file (e.g., styles.xml) to obtain style information (e.g., font, size, etc.) of specific text and use it to extract headings.

[0089] As another example, the system (10) can analyze the internal file structure (or data structure) of the target document to extract non-text document elements corresponding to images. For instance, if the target document is a 'Word document', the system (10) can extract each image from a folder where image files are stored within the internal file structure.

[0090] Referring again to Fig. 3, the explanation will be provided.

[0091] In step S34, the layout of a non-edited document (or target document) is analyzed using a generative language model based on the extracted document elements. For example, the system (10) can derive information regarding one or more layout elements by performing layout analysis using a generative language model (21) and / or another generative language model. Here, a layout element can be understood as a concept encompassing a document element that reflects layout information (e.g., location, role, document structure, etc.) and other layout elements (e.g., orientation of the document). Additionally, the correspondence between the document elements extracted in the previous step (S33) and the layout elements may be a one-to-one relationship, a one-to-many relationship, or a many-to-many relationship depending on the document structure.

[0092] For ease of understanding, step S34 will be further explained with reference to FIGS. 5 to 7.

[0093] FIG. 5 is an exemplary drawing for explaining the detailed process of step S34, illustrating the process of layout analysis being performed by the system (10).

[0094] As illustrated in FIG. 5, the system (10) can perform layout analysis by inputting a non-edited document image (51) and a prompt (52) into a generative language model (21). As described above, the generative language model (21) may be a model that further possesses the ability to understand images, such as a VLM.

[0095] Specifically, the system (10) can configure (or generate) a prompt (52) based on document element information (53) and a set of instructions (54), etc.

[0096] The document element information (53) may include, without limitation, information regarding various document elements extracted in the above-described step S33. Examples of such information may include the type of document element (e.g., text, image, table, etc.), value, structure information, etc., but are not limited thereto.

[0097] The set of instructions (54) may include, for example, instructions requesting a layout analysis of a non-edited document by referring to a non-edited document image (51) and document element information (53), instructions for enforcing an output format for the layout analysis result (56) (see FIG. 6, e.g., a format defined based on JSON), but is not limited thereto. For example, extractable candidate layout elements may be predefined in the output format.

[0098] In some cases, the prompt (52) may include additional examples related to layout analysis, etc.

[0099] Next, the system (10) can input the corresponding prompt (52) along with the non-edited document image (51) into the generative language model (21). Then, the generative language model (21) accurately analyzes the layout of the non-edited document by referring to the non-edited document image (51) and document element information (53), and generates an output (55) containing an analysis result (56). The analysis result (56) may include, for example, information about document elements present in the non-edited document in an identifiable form.

[0100] Refer to FIG. 6 for an example in which the output format of the layout analysis result (56) is defined in a JSON (JavaScript Object Notation) structure. In FIG. 6, the symbol ' / / ' represents an explanatory comment.

[0101] For reference, the inventors of the present disclosure have confirmed that information regarding each layout element (e.g., header, footer, document title, heading, document orientation, etc.) constituting a non-edited document (or target document) is accurately derived through a generative language model. In this case, the derived information can be understood to encompass not only the formal layout of the document but also information regarding its logical structure.

[0102] FIG. 7 illustrates a layout analysis process in the case where a non-edited document consists of multiple pages. In the following description, for clarity of the disclosure, 'Page 1' and 'Page 2' will be referred to as 'Page 1' and 'Page 2', respectively. Page 1 may be understood as corresponding to, for example, any page, and Page 2 may be understood as the page following Page 1.

[0103] As illustrated in FIG. 7, when a non-edited document consists of multiple pages, the system (10) can perform sequential layout analysis on a page-by-page basis.

[0104] Specifically, the system (10) can input a first prompt (72) containing document element information of the first page and a first page image (71) into a generative language model (21) to obtain a layout analysis result (73, hereinafter referred to as the 'first layout analysis result') for the first page.

[0105] Next, the system (10) may configure the second prompt (75) to include at least some of the document element information of the second page and the first layout analysis result (73). For example, the system (10) may add information regarding the headings of the first page (i.e., the previous page) (e.g., headings at each level such as subheadings, paragraph headings, etc.), headers, footers, and tables located at the bottom of the first page (e.g., tables spanning multiple pages) to the second prompt (75). This context / structural information of the previous page may contribute to improving the accuracy of the layout analysis of the next page.

[0106] In the example described above, at least some of the first layout analysis results (73) may also be reflected in prompts corresponding to subsequent pages of the second page (e.g., next page, the page after that, etc.). That is, at least some of the analysis results of the previous page may be cumulatively reflected in the prompts of subsequent pages. For example, information indicating the overall structure of the document, such as headings, may continue to be utilized in the layout analysis prompts of each of the subsequent pages.

[0107] Next, the system (10) can input the second prompt (75) and the second page image (74) into the generative language model (21) to obtain a layout analysis result for the second page (76, hereinafter referred to as the 'second layout analysis result').

[0108] The above-described process can be performed repeatedly until the last page of the non-edited document is reached.

[0109] Up to this point, the detailed process of the layout analysis step S34 has been described with reference to FIGS. 5 to 7. As described above, by providing a prompt containing document element information and a non-edited document image to a generative language model (e.g., 21), the layout of the non-edited document can be accurately analyzed. Furthermore, in the process of sequentially performing layout analysis on a page-by-page basis, the accuracy of the layout analysis for the non-edited document can be further improved by reflecting at least some of the layout analysis results of the previous page in the prompt of the next page.

[0110] Referring again to Fig. 3, the explanation will be provided.

[0111] In step S35, based on the layout analysis results, processed text corresponding to the target document is generated using a generative language model. For example, the system (10) can generate processed text by inputting a parsing prompt based on the layout analysis results into the generative language model (21) and / or another generative language model. Here, parsing may refer to the process of extracting each document element into a more structured form and reconstructing the extraction results. As a more specific example, the system (10) can directly generate processed text in Markdown format through the generative language model (21). As another example, the system (10) can generate initial processed text in another structured format (e.g., JSON format) through the generative language model (21) and convert the initial processed text into final processed text in Markdown format through post-processing. For such examples, further reference should be made to the descriptions in FIGS. 11 and FIGS. 12.

[0112] Hereinafter, the detailed process of step S35 will be further explained with reference to FIGS. 8 to 10.

[0113] FIG. 8 is an exemplary diagram for explaining the detailed process of step S36, illustrating the process of document parsing performed by the system (10). FIG. 8 illustrates a case where document parsing is performed through a generative language model (21) used for layout analysis, but the scope of the present disclosure is not limited thereto. The system (10) may perform document parsing using other generative language models as needed. Additionally, the system (10) may perform document parsing and / or layout analysis using at least one of the generative language models other than the generative language model (21).

[0114] As illustrated in FIG. 8, the system (10) can perform document parsing by inputting a non-edited document image (81) and a prompt (82) into a generative language model (21).

[0115] Specifically, the system (10) can configure (or generate) a prompt (82) based on document element information (83), layout analysis results (84), and a set of instructions (85). However, in some cases, the prompt (82) may be configured with the document element information (83) excluded.

[0116] The document element information (83) may include, without limitation, information regarding various document elements extracted in the above-described step S33.

[0117] For the layout analysis results (84), please refer to the above description and the example in Fig. 6.

[0118] The set of instructions (85) may include, for example, instructions requesting parsing of a non-edited document by referring to a non-edited document image (81), a layout analysis result (84) and / or document element information (83), instructions for enforcing an output format for a document parsing result (87) (e.g., a format defined based on JSON), but is not limited thereto.

[0119] In some cases, the prompt (82) may include additional parsing-related examples.

[0120] In some embodiments, the prompt (82) may be configured by dynamically combining element parsing prompts (e.g., pre-prepared system prompts) corresponding to each document element included in the layout analysis result (84). For example, as illustrated in FIG. 9, let us assume that a set of prompts (91) including a basic parsing prompt (92) and element parsing prompts (93-1 to 93-N) is prepared in advance. Here, the basic parsing prompt (92) may mean, for example, a prompt containing general parsing-related instructions (or information), and the element parsing prompt (e.g., 93-1) may mean a prompt containing parsing-related instructions specialized for a specific document element. In this case, the system (10) may obtain (or select) element parsing prompts corresponding to each document element present in the non-edited document from the prompt set (91) based on the layout analysis result (84) (e.g., a first element parsing prompt corresponding to the first document element and a second element parsing prompt corresponding to the second document element, such as 93-1, 93-4), and may construct a prompt (82) by combining these with a basic parsing prompt (92). As described above, the system (10) may further include the layout analysis result (84) and document element information (83), etc., in the prompt (82). According to these embodiments, the problem of reduced parsing accuracy and hallucination caused by the inclusion of parsing-related instructions for document elements not present in the non-edited document in the prompt can be effectively prevented.

[0121] Next, the system (10) can input the corresponding prompt (82) along with the non-edited document image (81) into the generative language model (21). Then, the generative language model (21) generates an output (86) containing a document parsing result (87) by comprehensively considering the non-edited document image (81), document element information (83), and layout analysis result (84).

[0122] The document parsing result (87) may correspond to the processed text of step S35, or it may be converted into processed text after post-processing. This post-processing method will be explained in detail later with reference to FIGS. 11 and FIGS. 12.

[0123] Figure 10 illustrates the document parsing process when a non-edited document consists of multiple pages.

[0124] As illustrated in FIG. 10, when a non-edited document consists of multiple pages, the system (10) can perform sequential document parsing on a page-by-page basis.

[0125] Specifically, the system (10) can obtain a parsing result (103, hereinafter referred to as the 'first parsing result') for the first page by inputting a first prompt (102), which includes document element information and layout analysis results of the first page, and a first page image (101) into a generative language model (21). As described above, the first prompt (102) may be generated, for example, by dynamically combining necessary element parsing prompts, but the scope of the present disclosure is not limited thereto.

[0126] Next, the system (10) may configure a second prompt (105) to include at least a portion of the first parsing result (103) and / or the first layout analysis result, as well as information on document elements of the second page. For example, the system (10) may add parsing results regarding the headings (e.g., subheadings, paragraph headings, headings of each level), headers, footers, and tables located at the bottom of the first page (e.g., tables spanning multiple pages) of the first page (i.e., the previous page) to the second prompt (105). This context / structural information of the previous page may contribute to improving parsing accuracy for the next page.

[0127] In the example described above, at least a portion of the first parsing result (103) and / or the first layout analysis result may also be reflected in the prompts corresponding to subsequent pages (e.g., next page, the page after that, etc.). For example, information indicating the overall structure of the document, such as headings, may continue to be utilized in the parsing prompts of each of the subsequent pages.

[0128] Next, the system (10) can input the second prompt (105) and the second page image (104) into a generative language model (21) to obtain a parsing result for the second page (106, hereinafter referred to as the 'second parsing result').

[0129] Up to now, the detailed process of the processed text generation step S35 has been described with reference to FIGS. 8 to 10. As described above, by providing a non-edited document image and a parsing prompt including the layout analysis result of the non-edited document together to a generative language model (e.g., 21), parsing of the non-edited document (or target document) can be performed precisely.

[0130] Hereinafter, post-processing methods according to some embodiments of the present disclosure will be described with reference to FIG. 11 and FIG. 12.

[0131] FIG. 11 is an exemplary flowchart illustrating a post-processing method. However, this is merely an exemplary embodiment for achieving the purpose of the present disclosure, and it is understood that some steps may be added or deleted as necessary.

[0132] As illustrated in FIG. 11, the post-processing method according to the embodiments may begin at step S111, which involves obtaining an initial processed text output by a generative language model. Here, the initial processed text can be understood as corresponding to the document parsing results described above (e.g., 87, 103, etc.). For example, the system (10) may obtain an initial processed text in JSON format output by a generative language model (21). Since the JSON format has a simple structure consisting of key-value pairs, it has the advantage of relatively low probability of syntax errors occurring by the generative language model (21).

[0133] In step S112, the initial processed text is converted into tag-formatted text. For example, the system (10) can convert the initial processed text in JSON format into a specified tag format (e.g., custom tag format, etc.). This conversion process can also serve to verify format or syntax errors before final conversion into Markdown format.

[0134] Meanwhile, in the preceding example, the system (10) may further perform post-processing to reinforce the data / information of non-text document elements so that non-text document elements can be more accurately reflected in the passage search process. This reinforcement processing (or post-processing) may be performed at any point after the document element extraction step S33, and the result of the processing may be utilized in the prompt for layout analysis and / or the parsing prompt. That is, this reinforcement processing may be performed independently of the post-processing method illustrated in FIG. 11.

[0135] For example, the system (10) can add tag items and their values ​​corresponding to the summary, table header, keywords, etc. of the table to enhance table data / information, and process the data of each cell in YAML format. A specific example of this is illustrated in FIG. 12. As illustrated in FIG. 12, the upper table (121) can be converted into the lower custom tag format text (122) through parsing and post-processing (i.e., enhancement processing). Additionally, as described below, the tag format text (122) can be converted back into Markdown format and included in processed text (or passage), and as a result, the table-related search function can be enhanced when providing knowledge services.

[0136] As another example, the system (10) may add tag items corresponding to captions, keywords, OCR text, etc. to enhance image data / information. A specific example of this is illustrated in FIG. 13. As illustrated in FIG. 13, the upper image (131) can be converted into the lower custom tag format text (132) through a post-processing process (i.e., enhancement processing). Additionally, as described below, the tag format text (132) can be converted back into a Markdown format and included in processed text (or passage), thereby enhancing image-related search functions when providing knowledge services.

[0137] As another example, the system (10) can generate a base numeric table by inversely calculating the chart to reinforce chart data / information, and process it in the same way as the table data described above. Any specific method of inverse calculation is acceptable. Specific examples of this are illustrated in FIGS. 14a and 14b. As illustrated in FIGS. 14a and 14b, a numeric table is generated through an inverse calculation process on the chart (141), and the numeric table can be converted into tag-format text (142) in the same way. Additionally, as described below, the tag-format text (142) can be converted back into Markdown format and included in processed text (or passage), and as a result, the chart-related search function can be enhanced when providing knowledge services.

[0138] As another example, the system (10) can enhance flowchart data / information using UML (Unified Modelling Language, e.g., PlantUML, etc.) representations. A specific example of this is illustrated in FIG. 15a. As illustrated in FIG. 15a, a flowchart (151) extracted from a non-edited document (or target document) can be converted into UML code (152, i.e., UML-based text). The UML code (152) can be included in the processed text (or passage) as is without separate format conversion, and as a result, the flowchart-related search function can be enhanced when providing knowledge services. FIG. 15b illustrates a result (153) of the UML code (152) rendered on a user terminal. For example, when a flowchart (151) is referenced when providing a knowledge service (e.g., when a passage containing UML code (152) is searched and referenced when generating an answer), a flowchart (153) generated by rendering the UML code (152) can be displayed on a user terminal.

[0139] As another example, the system (10) can enhance mathematical data / information using LaTeX (Lamport TeX) expressions. A specific example of this is illustrated in FIG. 16a. As illustrated in FIG. 16a, a mathematical formula (161) extracted from a non-edited document (or target document) can be converted into LaTeX code (162, i.e., LaTeX-based text). The LaTeX code (162) can be included in the processed text (or passage) as is without separate format conversion, and as a result, the search function related to mathematical formulas can be enhanced when providing knowledge services. FIG. 16b illustrates a result (163) of the LaTeX code (162) rendered on a user terminal. For example, when a formula (161) is referenced during the provision of a knowledge service (e.g., when a passage containing LaTeX code (162) is searched and referenced during the generation of an answer), the formula (163) generated by rendering the LaTeX code (162) can be displayed on the user terminal.

[0140] Referring again to FIG. 11, the following is explained: In step S113, it is determined whether the text has been successfully converted into a tag format. If it has been successfully converted, step S114 may be performed. If not, the processed text generation step S35 may be performed again, or other measures may be taken.

[0141] In step S114, the tag format text is converted into final processed text. For example, the system (10) can convert the tag format text into final processed text in Markdown format.

[0142] Up to this point, post-processing methods according to several embodiments of the present disclosure have been described with reference to FIGS. 11 and 12. As described above, initial processed text output by a generative language model (e.g., 21) is converted into tag-format text, and if the conversion is successful, the tag-format text can be converted into final processed text in Markdown format. In this case, syntax error problems that may occur in Markdown format can be prevented in advance. For example, syntax error problems that may frequently occur when a generative language model directly outputs processed text in Markdown format can be effectively resolved.

[0143] In addition, during the process of converting the initial processed text into tag-formatted text, various data / information regarding non-text document elements may be augmented and included in passages and / or their metadata. In this case, as non-text document elements are reflected in the passage search process, the search accuracy for related passages can be significantly improved.

[0144] Referring again to Fig. 3, the explanation will be provided.

[0145] In step S36, the processed text is divided into multiple passages as it is chunked. For example, the system (10) can chunk the processed text using the maximum passage length and document structure information included in the processed passage. Any specific chunking method may be used.

[0146] In step S37, each of the plurality of passages is mapped to a passage embedding and stored in the passage DB. For example, the system (10) may generate a passage embedding corresponding to each of the passages through a text embedding model and map each passage to the corresponding embedding and store it in the passage DB (11). At this time, the system (10) may embed the passage content (or original text), document title, summary of the passage, heading, etc. together, but the scope of the present disclosure is not limited thereto.

[0147] In some embodiments, metadata for each passage is generated and may be further stored in the passage DB (11). The metadata of a passage may include information such as headings, summaries, URIs, titles, keywords, etc., but the scope of the present disclosure is not limited thereto. The URI of a passage may, for example, point to the storage location of the non-edited document to which the passage belongs, and may include page-related fragments. Further reference to the description in FIG. 18 is made for this. FIG. 17 illustrates the content and metadata of a passage in the form of a single passage record. The illustrated metadata may also be used for passage searching.

[0148] Up to this point, a document processing method according to several embodiments of the present disclosure has been described with reference to FIGS. 3 through 17. As described above, a target document is converted into a non-edited document, and non-text document elements (e.g., images, charts, tables, diagrams, formulas, flowcharts, etc.) can be extracted from the non-edited document. In this case, various non-text document elements that are difficult to extract from the original target document can be extracted accurately and easily.

[0149] In addition, the target document can be converted into a non-editable PDF format with excellent compatibility, and a URI pointing to its storage location can be generated. Furthermore, if passages belonging to the target document are utilized to generate answers in a knowledge service, the source of the answer can be accurately presented using the corresponding URI, and the source document can be easily viewed without the need for a dedicated document program.

[0150] In addition, since the URI associated with each passage includes page information of the document to which the passage belongs, detailed page-level source information can be presented to users of the knowledge service. Accordingly, users can directly access the source page of the answer through the corresponding URI.

[0151] In addition, by controlling the generative language model (e.g., 21) to separate layout analysis and document parsing tasks so that they are performed independently, the accuracy and processing stability of each task can be significantly improved.

[0152] In addition, by reflecting at least some of the layout analysis results of the previous page in the layout-related prompts of the next page, page-level layout analysis of a non-edited document (or target document) can be performed more accurately.

[0153] In addition, by reflecting at least some of the parsing results from the previous page in the parsing prompt of the next page, page-by-page parsing operations for a non-edited document (or target document) can be performed more precisely.

[0154] Hereinafter, a method for providing knowledge services according to some embodiments of the present disclosure will be described with reference to FIG. 18.

[0155] FIG. 18 illustrates the detailed process of the method performed by the knowledge service system (20), and the following description may be understood as a specific example of the use of the passage DB (11).

[0156] As illustrated in FIG. 18, the knowledge service system (20) can provide RAG-based knowledge services using a passage DB (11) and a generative language model (181). The knowledge service system (10) may provide knowledge services to multiple users, or it may provide customized knowledge services (e.g., personal assistant services, etc.) specialized for specific users.

[0157] Specifically, the knowledge service system (20) can obtain related passages by receiving a query (182) from a user terminal (180) and using it to search the passage DB (11). For example, the knowledge service system (20) can obtain the top K (where K is a natural number greater than or equal to 1) related passages by searching the passage DB (11) based on embedding similarity with the user query (182).

[0158] Next, the knowledge service system (20) can construct a prompt based on the user query (182) and the retrieved related passages, and input this into the generative language model (181) to generate an answer (183).

[0159] For example, a knowledge service system (20) can obtain a final answer (183) by generating an initial answer through a generative language model (181) and performing post-processing on it. Here, post-processing may include adding (or inserting) a URI pointing to the source (185) of the answer (183), adding a URI pointing to an image (184) associated with the query (182) and / or the answer (183), reinforcing the content, verifying the content, modifying the style, adjusting the tone, etc., but the scope of this disclosure is not limited thereto. Such post-processing may be performed based on a separate algorithm (or logic) or may be performed through the generative language model (181) or another generative language model. As another example, the knowledge service system (20) may control the generative language model (181) to directly generate the answer (183) in a form with post-processing applied through a prompt.

[0160] The source information (185) of the answer (183) may, for example, refer to the document to which the passage (i.e., the retrieved associated passage) referenced by the generative language model (181) during the answer generation process belongs, or the information of the document. The knowledge service system (20) may obtain the URI of the non-edited document from the metadata of the referenced passage and use it to provide the source information (185) of the answer (183). The specific method of providing the source information (185) may be any method. For example, the knowledge service system (20) may provide the URI itself, the title of the document containing the URI, the document icon, etc., as source information (185), or may directly provide the source document pointed to by the URI.

[0161] In some embodiments, the URI may be configured to include a fragment that specifies a specific page associated with the reference passage (e.g., the page to which the passage belongs) (see underlined part). In this case, detailed source information at the page level may be presented to the user, and the user may be able to immediately identify (or view) the document page closely associated with the answer (183) via the URI.

[0162] The image (184) may refer to an image element or document element associated with the query (182) and / or answer (183) that is stored in an image format. For example, the image (184) may represent a non-text document element (e.g., image, table, chart, diagram, etc.) referenced by the generative language model (181) during the answer generation process. The knowledge service system (20) may provide the answer to the user by including a URI pointing to the storage location of the image (184). For example, as illustrated, an image insertion syntax in Markdown format containing the URI may be included in the answer (183) and transmitted to the user terminal (180), and as a result, the image (184) of the associated non-text document element may be displayed on the user terminal (180) along with other content of the answer (183). That is, the image (185) of the non-text document element may be displayed together on the user terminal (180) during the process of providing the answer (183).

[0163] Up to this point, a method for providing knowledge services according to some embodiments of the present disclosure has been described with reference to FIG. 18. As described above, the source of the answer can be accurately presented using a URI included in the metadata of a reference passage, and detailed source information at the page level can also be provided to the user.

[0164] In addition, by including a URI pointing to an image of a non-text document element in the answer, a richer answer can be provided to the user.

[0165] Hereinafter, with reference to FIG. 19, an exemplary computing device (190) capable of implementing the system (10) described above will be described.

[0166] FIG. 19 is an exemplary hardware configuration diagram showing a computing device (190).

[0167] As illustrated in FIG. 19, a computing device (190) may include one or more processors (191), a bus (193), a communication interface (194), a memory (192) for loading a computer program (196) executed by the processor (191), and a storage (195) for storing the computer program (196). However, FIG. 19 illustrates only the components related to the embodiments of the present disclosure. Therefore, a person skilled in the art to which the present disclosure belongs will understand that other general-purpose components may be included in addition to the components (191 to 196) illustrated in FIG. 19. That is, the computing device (190) may include various additional components in addition to the components (191 to 196) illustrated in FIG. 19. Furthermore, depending on the case, the computing device (190) may be configured in a form in which some of the components (191 to 196) illustrated in FIG. 19 are omitted. Below, each component of the computing device (190) is described.

[0168] The processor (191) can control the overall operation of each component of the computing device (190). The processor (191) may be configured to include at least one of a CPU (Central Processing Unit), MPU (Micro Processor Unit), MCU (Micro Controller Unit), GPU (Graphic Processing Unit), NPU (Neural Processing Unit), TPU (Tensor Processing Unit), VPU (Vision Processing Unit), APU (Accelerated Processing Unit), or any other type of processor well known in the art of the present disclosure. Additionally, the processor (191) may perform operations for at least one application or program to execute specific operations / steps / methods. The computing device (190) may have one or more processors.

[0169] Next, the memory (192) may store various data, commands and / or information. The memory (192) may load a computer program (196) from storage (195) to execute specific operations / steps / methods. The memory (192) may be implemented as volatile memory such as RAM, but the technical scope of the present disclosure is not limited thereto.

[0170] The bus (193) can provide communication functions between components of the computing device (190). The bus (193) can be implemented as various types of buses, such as an address bus, a data bus, and a control bus.

[0171] The communication interface (194) can support wired and wireless internet communication of the computing device (190). Additionally, the communication interface (194) may support various communication methods other than internet communication. To this end, the communication interface (194) may be configured to include a communication module well known in the art of the present disclosure.

[0172] Storage (195) may store one or more computer programs (196) non-temporarily. Storage (195) may be configured to include non-volatile memory such as ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, a hard disk, a removable disk, or any form of computer-readable recording medium well known in the art to which this disclosure belongs.

[0173] A computer program (196) may include instructions that cause a processor (191) to perform specific operations / steps / methods when loaded into memory (192). That is, the processor (191) can perform specific operations / steps / methods by executing the loaded instructions.

[0174] For example, a computer program (196) may include instructions for performing operations such as converting a target document into a non-editing document, extracting one or more document elements constituting the target document based on the non-editing document, inputting a parsing prompt based on one or more document elements into a generative language model to obtain processed text corresponding to the target document, and chunking the processed text to divide it into multiple passages.

[0175] As another example, a computer program (196) may include instructions to perform at least some of the operations / steps / methods described with reference to FIGS. 1 through 17.

[0176] As illustrated, a system (10) according to some embodiments of the present disclosure can be implemented through a computing device (190).

[0177] In some cases, the computer program (196) may include instructions for performing at least some of the operations / steps / methods described with reference to FIG. 18. In such cases, a knowledge service system (20) according to some embodiments of the present disclosure may be implemented through a computing device (190).

[0178] Meanwhile, in some embodiments, the computing device (190) illustrated in FIG. 19 may refer to a virtual machine implemented based on cloud technology. For example, the computing device (190) may be a virtual machine running on one or more physical servers included in a server farm. In this case, at least some of the processor (191), memory (192), and storage (195) illustrated in FIG. 19 may be virtual hardware, and the communication interface (194) may also be implemented as a virtualized networking element such as a virtual switch.

[0179] Up to now, with reference to FIG. 19, an exemplary computing device (190) capable of implementing a system (10) according to some embodiments of the present disclosure has been described.

[0180] Various embodiments of the present disclosure and effects according to those embodiments have been described with reference to FIGS. 1 through 19. The effects according to the technical concept of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below.

[0181] Furthermore, just because the above embodiments describe a plurality of components being combined into one or operating in combination, the technical concept of the present disclosure is not necessarily limited to these embodiments. That is, within the scope of the purpose of the technical concept of the present disclosure, all such components may be selectively combined into one or more combinations to operate.

[0182] The technical concept of the present disclosure described above may be implemented as computer-readable code on a computer-readable recording medium (e.g., a non-transitory recording medium). A computer program recorded on a computer-readable recording medium may be transmitted to another computing device via a network such as the Internet and installed on said computing device, thereby being used on said computing device.

[0183] Although operations are depicted in a specific order in the drawings, it should not be understood that the operations must necessarily be executed in the specific order depicted or in a sequential order, or that all depicted operations must be executed to obtain the desired result. In certain situations, multitasking and parallel processing may be advantageous. Although various embodiments of the present disclosure have been described above with reference to the attached drawings, those skilled in the art will understand that the technical concept of the present disclosure may be implemented in other specific forms without altering the technical concept or essential features thereof. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. The scope of protection of the present disclosure shall be interpreted by the claims below, and all technical concepts within the equivalent scope shall be interpreted as being included within the scope of rights of the technical concept defined by the present disclosure. Explanation of the symbols

[0184] 10: Document Processing System 11: Passage DB (passage database) 20: Knowledge Service System 21, 181: Generative Language Model 180: User terminal 190: Computing device 191: Processor 192: Memory 193: Bus 194: Communication Interface 195: Storage 196: Computer Programs

Claims

Claim 1 A method for processing documents, comprising: a step of converting a target document into a non-editable document, performed by at least one processor; a step of extracting one or more document elements constituting the target document based on the non-editable document; a step of analyzing the layout of the non-editable document using a generative language model; a step of inputting a parsing prompt based on the analysis result of the extracted one or more document elements and the layout into at least one of the generative language model or another generative language model to obtain processed text corresponding to the target document; and a step of chunking the processed text to divide it into a plurality of passages. Claim 2 A document processing method according to claim 1, wherein the above-mentioned non-edited document is a document in PDF (Portable Document Format) format. Claim 3 A document processing method according to claim 1, wherein the target document is a document in a file format configured such that at least one non-text document element is displayed through rendering by a dedicated document program. Claim 4 A document processing method according to claim 1, wherein the step of extracting one or more document elements comprises: a step of extracting one or more non-text objects from the non-edited document and placing them in a new non-edited document; and a step of extracting non-text document elements by applying an outline detection technique or an object detection technique to an image of the new non-edited document. Claim 5 A document processing method according to claim 1, wherein the step of extracting one or more document elements includes the step of extracting an area corresponding to a specific non-text object in an image of a non-edited document using location information and size information of a specific non-text object obtained from internal data of the target document. Claim 6 A document processing method according to claim 1, wherein one or more extracted document elements include non-text document elements extracted from the non-edited document, and the parsing prompt is further based on text document elements extracted from the target document. Claim 7 delete Claim 8 A document processing method according to claim 1, wherein the step of analyzing the layout of the non-edited document comprises the step of inputting a prompt containing information of one or more extracted document elements and an image of the non-edited document into the generative language model to perform a layout analysis of the non-edited document. Claim 9 A document processing method according to claim 1, wherein the non-edited document includes a first page and a second page which is the next page of the first page, and the step of analyzing the layout of the non-edited document comprises: a step of obtaining a first layout analysis result for the first page by inputting a first prompt including document element information of the first page and an image of the first page into the generative language model; a step of configuring a second prompt to include at least a portion of the first layout analysis result and document element information of the second page; and a step of obtaining a second layout analysis result for the second page by inputting the second prompt and an image of the second page into the generative language model. Claim 10 A document processing method according to claim 1, wherein the analysis result of the layout includes information regarding a first document element and a second document element reflecting layout information, and the step of obtaining the processed text comprises: obtaining a first element parsing prompt corresponding to the first document element and a second element parsing prompt corresponding to the second document element from a pre-prepared set of prompts; configuring the parsing prompt by combining the first element parsing prompt and the second element parsing prompt; and inputting the parsing prompt into the at least one model. Claim 11 A document processing method according to claim 1, wherein the non-edited document includes a first page and a second page which is the next page of the first page, and the analysis result of the layout includes a first layout analysis result for the first page and a second layout analysis result for the second page, and the step of obtaining the processed text includes: a step of obtaining a first parsing result for the first page by inputting a first parsing prompt including the first layout analysis result and an image of the first page into the at least one model; a step of configuring a second parsing prompt such that at least a portion of the first parsing result and the first layout analysis result is included; and a step of obtaining a second parsing result for the second page by inputting the second parsing prompt and an image of the second page into the at least one model. Claim 12 A document processing method according to claim 1, wherein the step of obtaining the processed text comprises: obtaining an initial processed text output through at least one model; converting the initial processed text into tag format text; and, if the initial processed text is successfully converted into the tag format text, converting the tag format text into the processed text in markdown format. Claim 13 A document processing method according to claim 1, wherein a specific passage among the passages is stored in a passage database along with metadata, the metadata includes a Uniform Resource Identifier (URI) of the non-edited document, and when an answer to a user query is provided based on the specific passage, source information of the answer is provided along with the URI. Claim 14 A document processing method according to claim 13, wherein the specific passage is associated with a specific page of the non-edited document, and the URI includes a fragment specifying the specific page. Claim 15 A document processing method according to claim 1, wherein one or more extracted document elements include non-text document elements, an image of the non-text document element is stored at a specific location, and when an answer associated with the non-text document element is provided in response to a user's query: a Uniform Resource Identifier (URI) pointing to the specific location is included in the answer, and during the process of providing the answer, an image of the non-text document element is displayed on the user's terminal through the URI. Claim 16 A document processing system comprising: one or more processors; and memory for storing a computer program executed by the one or more processors, wherein the computer program comprises: an operation of converting a target document into a non-editable document; an operation of extracting one or more document elements constituting the target document based on the non-editable document; an operation of analyzing the layout of the non-editable document using a generative language model; an operation of inputting a parsing prompt based on the analysis result of the extracted one or more document elements and the layout into at least one of the generative language model or another generative language model to obtain processed text corresponding to the target document; and instructions for an operation of chunking the processed text and dividing it into a plurality of passages. Claim 17 delete Claim 18 A document processing system according to claim 16, wherein a specific passage among the above passages is stored in a passage database along with metadata, said metadata includes a Uniform Resource Identifier (URI) of said non-edited document, and when an answer to a user query is provided based on said specific passage, said URI pointing to said non-edited document is provided together as source information of said answer. Claim 19 A computer program stored on a non-transitory computer-readable recording medium to execute the steps of: converting a target document into a non-editable document by combining with a computer; extracting one or more document elements constituting the target document based on the non-editable document; analyzing the layout of the non-editable document using a generative language model; inputting a parsing prompt based on the analysis results of the extracted one or more document elements and the layout into at least one of the generative language model or another generative language model to obtain processed text corresponding to the target document; and chunking the processed text and dividing it into a plurality of passages.

Images