Method and apparatus for processing document realted tasks using llm

Abstract

An LLM-based task processing method and system are provided, which identify a region of interest in a target document, generate reference data corresponding to the region of interest based on the analysis results of the region of interest, generate target data by inputting a prompt for processing a task related to the reference data and the target document into a Large Language Model (LM), and process the task using the target data.

Description

Technology Field

[0001] The present disclosure relates to a method and apparatus for processing document-related tasks using an LLM. More specifically, it relates to a method for automatically analyzing specification documents of steel products and extracting specification information using an artificial intelligence large language model (LLM), and an apparatus to which the method is applied. Background Technology

[0002] Specification documents for steel products contain various technical information, such as physical properties, manufacturing methods, and test conditions, and serve as important standards during the actual manufacturing and order fulfillment processes. Based on these documents, tasks can be performed, such as determining whether a specific steel product meets requirements or deriving optimal standards by comparing multiple documents.

[0003] Traditionally, the method used to analyze steel standard documents involved specialized personnel in the field directly reviewing the documents and manually organizing the necessary information. However, this approach requires significant time and effort because the information within the documents is scattered across various formats, such as tables, body text, and footnotes. Furthermore, analysis results are prone to variation depending on the skill level of the person in charge, and even when analyzing the same document, omissions of information or interpretation errors may occur.

[0004] To address these issues, a procedure in which two or more personnel cross-review the same document is sometimes utilized; however, this approach has limitations, such as increased labor costs and longer overall analysis times. Consequently, there is a growing need for automation and intelligent technologies capable of processing steel specification documents more quickly and accurately. Prior art literature

[0005] Korean Patent Publication No. 10-2025-0064953 (Published May 12, 2025) The problem to be solved

[0006] The technical problem that the present invention aims to solve is to provide a method and apparatus capable of automatically analyzing specification documents for steel products using an artificial intelligence large language model (LLM), and accurately extracting, outputting, and storing key specification information.

[0007] Another technical problem that the present invention aims to solve is to facilitate comparison and analysis of differences between documents by converting steel standard specification documents and order specification documents by manufacturer, which are described in various formats, into a unified format and outputting them.

[0008] Another technical problem that the present invention aims to solve is to provide an AI-based document analysis system that automates specification analysis tasks, which were previously performed manually by experts, thereby ensuring the accuracy and consistency of the analysis and reducing the overall analysis time and cost.

[0009] The technical problems of the present disclosure are not limited to those mentioned above, and other unmentioned technical problems will be clearly understood by a person skilled in the art of the present disclosure from the description below. means of solving the problem

[0010] An LLM-based task processing method according to one embodiment of the present disclosure for solving the above technical problem may include the steps of: identifying a region of interest in a target document; generating reference data corresponding to the region of interest based on an analysis result of the region of interest; generating target data by inputting a prompt for processing a task related to the reference data and the target document into a Large Language Model (LM); and processing the task using the target data.

[0011] In one embodiment, the target document may include at least one of a standard specification document by steel grade, a manufacturer's order specification document, a new standard document, and an arbitrary specification document established for manufacturing.

[0012] In one embodiment, the region of interest may be a region containing a table composed of a plurality of rows and columns, and the reference data may include structural information of the table and information written in each cell of the table.

[0013] In one embodiment, the reference data may further include HTML tag data corresponding to the table. In this case, the HTML tag data may be generated when the table includes a plurality of merged cells or when the number of cells is greater than or equal to a threshold value.

[0014] In one embodiment, the prompt may include at least one of information regarding the name of the specification item to be extracted within the target document, the output format of the specification item, and the output method of the specification item.

[0015] In one embodiment, the prompt may be determined based on at least one of the type of the target document and the type of the task.

[0016] In one embodiment, the task may be to extract information regarding the first specification from the target document. Additionally, the target data may be an HTML tag storing the structure and content of a table containing information regarding the first specification.

[0017] In one embodiment, the target document may include a first target document and a second target document. Additionally, the task may be processed by selecting only the specification item corresponding to the intersection among the specification items extracted from the first target document and the second target document, respectively.

[0018] In one embodiment, the target document may include a first target document and a second target document. Additionally, the task may be processed by selecting all specification items formed by a union among the specification items extracted from the first target document and the second target document, respectively.

[0019] In one embodiment, the target document may be a plurality of target documents, and the task may be processed by selecting the first specification item from the document with the highest priority according to a predefined priority among the target documents when the first specification item exists in common in the plurality of target documents. Effects of the invention

[0020] According to the present embodiment, by automatically analyzing steel specification documents, the accuracy of document analysis can be improved, and the time and cost consumed in document analysis can be reduced.

[0021] According to the present embodiment, the analysis results of steel order documents are structured and standardized in a unified format, thereby facilitating the comparison of key information between documents and allowing for the intuitive identification of differences and commonalities between documents from different manufacturers or specifications.

[0022] According to the present embodiment, by accurately providing information necessary for steel product development in response to a vast range of steel standard specifications, the system enables the development of superior products, thereby promoting the overall development of the steel industry.

[0023] The effects according to the technical concept of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below. Brief explanation of the drawing

[0024] FIG. 1 is an exemplary flowchart of a method for processing tasks related to steel specification documents using LLM according to some embodiments of the present disclosure. Figure 2 is an example diagram to explain some operations illustrated in Figure 1. FIG. 3 is an example diagram of the result of a task processing related to a steel specification document using LLM according to one embodiment of the present disclosure. FIG. 4 is a flowchart illustrating an LLM-based task processing method according to one embodiment of the present disclosure. FIGS. 5 to 7C are illustrative diagrams for explaining some operations illustrated in FIG. 4. FIGS. 8 to 16 are exemplary diagrams of a user input prompt and a task processing result according to the prompt input, according to some embodiments of the present disclosure. FIG. 17 is a hardware configuration diagram of a computing device according to some embodiments of the present disclosure. Specific details for implementing the invention

[0025] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the attached drawings. The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the attached drawings. However, the technical concept of the present invention is not limited to the following embodiments but can be implemented in various different forms. The following embodiments are provided merely to complete the technical concept of the present invention and to fully inform those skilled in the art of the scope of the present invention, and the technical concept of the present invention is defined only by the scope of the claims.

[0026] In describing the present disclosure, if it is determined that a detailed description of related known configurations or functions could obscure the essence of the invention, such detailed description is omitted.

[0027] Hereinafter, some embodiments of the present disclosure will be described with reference to the drawings.

[0028] FIG. 1 is an exemplary flowchart of a method for processing tasks related to steel specification documents using a Large Language Model (LM) according to some embodiments of the present disclosure.

[0029] As illustrated in FIG. 1, a user can input one or more target documents into an LLM-based task processing system (10a). At this time, the target documents are documents containing information about steel specifications that the user wants to analyze and extract, and may be at least one of a standard specification document by steel grade, a manufacturer's order specification document, a new standard document, and an arbitrary specification document set for manufacturing.

[0030] When a user inputs a target document to be analyzed, the LLM-based task processing system can identify a region of interest on each page of the input target document (10b). At this time, the region of interest is an area within the document that the user wants to analyze in detail, and may be at least one of a region where text is displayed (i.e., text area), a region where a table is displayed (i.e., table area), and a region where a picture is displayed (i.e., image area).

[0031] More specifically, an LLM-based task processing system can detect regions of interest containing tables by applying object detection or document layout analysis algorithms to each page of a target document input by a user. In this case, YOLO-based table detection models, paragraph separation models, and OCR post-processing technologies may be used together. Additionally, the method for identifying regions of interest within a document may vary depending on the document type (e.g., PDF, scanned image, Word, etc.), and if necessary, visual noise removal and contrast correction within the document may also be performed.

[0032] When a region of interest (e.g., a table region) is identified within a target document, the LLM-based task processing system can perform additional structural analysis on the identified region of interest and store the analysis results (i.e., reference data) in a vector database (10c). At this time, the structural analysis may include the number of rows and columns of the table, the cell merging status, whether there is a header hierarchy, etc., and may generate structured reference data along with the text data contained in each cell of the table.

[0033] Meanwhile, the aforementioned reference data may be expressed in various formats, such as key-value mapping (Dictionary), JSON structure, or HTML table structure, rather than simply listing text; in particular, when the visual representation of task processing results is important, it may be converted into and stored in the form of HTML tags. Additionally, the model for identifying table areas within a document and the model for analyzing the table structure may be models trained using different methods. In this case, the model for identifying table areas and the model for analyzing the table structure may be a publicly available AI-based network learning model or a separately trained network model.

[0034] In some embodiments, to improve the performance of identifying and analyzing table regions in target documents, tables typically described in steel-related documents to be analyzed may be used for training the network model. Additionally, to extract meaningful item names within the tables, language model-based text classification or Name Entity Recognition (NER) methods may also be used for training the network model.

[0035] In one embodiment, the region of interest may be a text region, and reference data extracted from the text region, either the text itself or the text in a summarized form, may be stored in a text list of a vector database (10c). In this case, the extraction of the text region may be performed using a commonly disclosed technique or by using a separately trained artificial intelligence-based network model. As those skilled in the art are likely already familiar with artificial intelligence-based network models for extracting text from documents, a detailed explanation thereof will be omitted.

[0036] In one embodiment, the region of interest may be a table region containing a table composed of one or more rows and columns, and reference data including structural information of the table and information written in each cell of the table may be generated by an artificial intelligence-based network model, and the generated reference data may be stored in a table list of a vector database. In this case, the network model may extract information of the text contained in each cell according to the row header and column header portions, output the text in the form of a string in an orderly manner so that it can be easily analyzed in an LLM, and store the output result in a table list of the vector database.

[0037] In one embodiment, HTML tag data containing structural information of the table may also be stored in the table list of the vector database. In this case, the HTML tag data may be generated only when the table contains multiple merged cells or when the number of cells constituting the table is greater than or equal to a threshold. In another embodiment, the HTML tag data may always be generated regardless of the type of the table.

[0038] Referring again to FIG. 1, the LLM-based task processing system can analyze the target document (e.g., steel document) and extract specification information (e.g., target data) desired by the user by inputting (10d) a prompt that specifies a vector database extracted from the target document, data to be analyzed by the user, and the output format of the analysis result into the LLM (10e). Subsequently, the specification information (e.g., target data) extracted through the LLM is output in the form desired by the user, thereby allowing the task related to the target document to be processed.

[0039] To elaborate, the above prompt is not merely a simple command string, but may include details such as the name of the specification item to be extracted, the output format (e.g., text summary, numerical display, extraction of the entire table, etc.), and the priority and conditions of the output target (e.g., specific steel grade, threshold value or higher, etc.). Furthermore, the above prompt may be determined differently depending on the type of target document, whether comparison between target documents is performed, and the type of task desired by the user.

[0040] In one embodiment, the prompt may be selected by the user from among a plurality of predefined prompts for each type of task, or it may be automatically selected by the LLM based on information about the type of task entered by the user and the type of the target document.

[0041] In one embodiment, if the content of the specification information that the user wants to obtain must include all items listed in multiple cells of a table, the prompt entered into the LLM may include content instructing the specification information to extract the entire table, save it in HTML format, and output it.

[0042] In one embodiment, if the content of the specification information that the user wishes to obtain must be indicated by a specific symbol or number, the prompt input to the LLM may include content instructing to extract and indicate only that number or symbol.

[0043] Meanwhile, information on each specification item output through the LLM can be stored in a dictionary structure with a key-value structure. In this case, the key is the item name of the specification, and the value can be extracted information of the specification. Additionally, the analysis results of the target document output through the LLM can be output in a predefined document format according to the user's selection.

[0044] In one embodiment, information on specification items extracted from a target document according to a predefined order and location can be output in a table or Excel format. In this case, if a single specification item is extracted as a table, the information on the specification item can be output as an HTML tag that preserves the structure and content of the table as is, and the entire table can be output when a user clicks the tag.

[0045] In some embodiments, the term 'target data' as used herein may refer to the results of information analysis generated by the LLM in response to user prompts, and may include text information regarding specification items, numerical values, structured table data, HTML format data of tables, etc. That is, target data is a result representing key information extracted from a target document, and can be understood as referring to the content itself, regardless of the output format in which the information is provided.

[0046] In addition, target data can be defined as encompassing the overall document analysis results generated through LLM, and can be understood as referring to the results of the document analysis task itself, regardless of the output format.

[0047] In some embodiments, depending on the user's task conditions and output settings, the target data may be output as simple text or numerical data, or generated as structured information in a tabular format and converted into a visual output format such as HTML or CSV. For some tasks, the target data may be used directly as an output result without separate subsequent processing, while for others, it may be output after undergoing an additional visualization process.

[0048] In summary, an LLM-based task processing system can identify regions of interest (e.g., tables, text) within a target document (e.g., steel specification document) and generate reference data (e.g., lists of strings, lists of tables) corresponding to the said regions of interest. Additionally, target data (e.g., information on the first specification item) can be generated as a result of inputting the reference data and a prompt for processing tasks related to the target document into the LLM. Subsequently, the LLM-based task processing system can use the said target data to process a target document-related task desired by the user (e.g., outputting information on the first specification item in a table or Excel format).

[0049] Meanwhile, as illustrated in FIG. 1 (10a), at the step of inputting target documents, the user can specify priority for each target document, input the type of steel to be analyzed and extracted, and select an Excel extraction version of the data output through the LLM. That is, the user can input multiple target documents into an LLM-based task processing system, specify priority among multiple target documents, and process a task of comparing multiple target documents.

[0050] In one embodiment, a user may input two or more target documents (e.g., specification documents) into an LLM-based task processing system, and the task processing system may analyze multiple specification documents simultaneously. At this time, the task processing system may store and output the analysis results of the two specification documents as a union, and may also output and store only the items corresponding to the intersection.

[0051] In one embodiment, when a user inputs and intends to analyze multiple target documents (i.e., a first specification document, a second specification document), the user may assign priorities among the multiple specification documents, and the LLM-based task processing system may prioritize the output and storage of specification item information in the target document with the highest priority (i.e., the first specification document). Additionally, if specification item information that does not exist in the target document with the highest priority (i.e., the first specification document) is extracted from a target document with a lower priority (i.e., the second specification document), the LLM-based task processing system may output and store the specification item information in the target document with the lower priority (i.e., the second specification document). At this time, the name of the target document from which the specification was extracted may also be output.

[0052] In one embodiment, when a user inputs a plurality of target documents into an LLM-based task processing system, the task processing system may compare the contents of specification item information analyzed within the plurality of target documents and present the differences between the target documents as a result of the comparison. That is, the task processing system may compare the differences regarding the information of identical specification items listed within the plurality of target documents and store and output the analysis results of the comparison.

[0053] In another embodiment, an LLM-based task processing system can receive multiple specification item information (e.g., first specification item information within a first target document, second specification item information within a second target document) generated by completing the analysis of a target document using LLM, and can compare and analyze the differences between the received specification item information.

[0054] Meanwhile, in the case of a standard document containing multiple steel grades, the user can use an LLM-based task processing system to selectively extract, save, and output only the standard information of a specific steel grade, and can also select and output and save only specific standard items.

[0055] Figure 2 is a diagram showing in more detail an exemplary process of identifying a region of interest in a target document and extracting the corresponding table structure (i.e., reference data) among the processes described in Figure 1.

[0056] As illustrated in FIG. 2, the target document (2a) input to the LLM-based task processing system is generally provided in PDF format or scanned image format, and the task processing system can first extract the bounding box coordinates of the area where the table is located on each page through a network model for table detection (2b) (2b). That is, when the image of the target document is input to the network model for table detection, location information of the table included in the target document can be output. At this time, filtering can be performed based on visual elements such as the thickness of the outline of the table included in the target document, the spacing between cells, and the matrix alignment status.

[0057] Afterwards, the internal structure of each table can be analyzed in detail through a table structure recognition model (2d) (2e). At this time, the analysis of the internal structure of the table may not be limited to simply distinguishing the number of rows and columns, but may include the presence or absence of merged cells, multi-layered header structure, logical dependency relationships between cells, etc.

[0058] For example, if the meaning of items such as specification names, reference values, and tolerance ranges varies depending on the cell location, an LLM-based task processing system may include a process of reconstructing them during LLM input by identifying the above relationships in the analysis process of the internal structure of the table above.

[0059] Once the analysis of the internal structure of the table is complete, the table may be structured in CSV or HTML format. In this case, the use of HTML tags is understood to allow the user to view the original table form within the target document or to visually render it. If the content of the table is long or spans multiple pages, the results of the internal structure analysis may be divided into table units, merged, or summarized and saved.

[0060] Meanwhile, non-table plain text areas can be stored in a vector database after undergoing separate sentence separation and semantic unit analysis. This text data can be utilized for full document summarization, context-based task definition, and prompt configuration.

[0061] For example, if a specific document's technical specification does not contain a table but the technical standard is described in sentences within the body text, the system can be designed to analyze those sentences and extract standard data based on specific keywords (e.g., steel grade, tensile strength, standard value, etc.). In such cases, the reliability of the extracted data can be measured by having models that analyze noun phrases within the sentences and models that extract numerical values ​​work together.

[0062] Subsequently, the tables and text within the target document can be generated in the form of reference data necessary for user-defined task processing related to the target document based on the analysis results of the document and stored in a database, and the stored reference data can be used in the target data generation stage, which is utilized in the task processing stage using LLM.

[0063] FIG. 3 is an example diagram showing the result of processing a user-defined task related to a steel specification document using LLM according to an embodiment of the present invention.

[0064] As illustrated in FIG. 3, the processing result (i.e., output result) of a task related to a target document (e.g., steel specification document) may be automatically generated according to the conditions of a prompt set in advance by the user.

[0065] More specifically, the processing result may be output in a table format comprising the names of specification items within the document (e.g., Chemical Composition (Al, B, C, Cr, etc.), Quality (Item, Anti-rust, Carbon Segregation, Decarburization, etc.) and their corresponding values. In this case, the output may be generated in an Excel file (.xlsx) or CSV format, and may be converted into an HTML structure in its original table form so that it can be visually viewed on a web browser.

[0066] In addition, supplementary information such as the name of the relevant document, the page number of the specification, and the source of the item can be included for each item, and if necessary, quality grades or judgment results (Quality, Conclusion) for the user's subsequent review can also be automatically inserted. Such structured output results can be utilized in various ways for purposes such as document-based order review, quality assurance response, and delivery condition review.

[0067] Furthermore, although not illustrated in FIG. 3, the output of the task processing result may be configured based on a single document, or it may be configured in a table format that analyzes multiple documents in parallel to compare differences by item. Additionally, if the entire specific table within the target document is determined to be an item to be analyzed, the entire table can be converted into an HTML tag-based structure and output. In this case, source information such as the filename, page number, and table number of the original document may be assigned to each cell to enhance traceability and reliability.

[0068] Up to now, with reference to FIGS. 1 to 3, examples of LLM-based task processing methods and processing results according to some embodiments of the present disclosure have been described. Hereinafter, with reference to FIGS. 4 and subsequent drawings, methods according to various embodiments of the present disclosure will be described in detail.

[0069] Each step of the methods described below may be performed by a computing device. In this case, the computing device may be a notebook, desktop, laptop, etc., but is not limited thereto and may include all types of devices equipped with computing functions. An example of a computing device will be described with reference to FIG. 17.

[0070] In other words, each step of the methods may be implemented by one or more instructions executed by a processor of a computing device. All steps included in these methods may be executed by a single physical computing device, but the first steps of the methods may be performed by a first computing device and the second steps of the methods may be performed by a second computing device. Hereinafter, the explanation will continue assuming that each step of the methods is performed by an LLM-based task processing system. However, for the convenience of explanation, the subject of operation for each step included in the methods may be omitted.

[0071] FIG. 4 is a flowchart illustrating an LLM-based task processing method according to one embodiment of the present disclosure. However, this is merely a preferred embodiment for achieving the purpose of the present disclosure, and it is understood that some steps may be added or deleted as needed.

[0072] As illustrated in FIG. 4, the LLM-based task processing method according to the present embodiment begins at step S100, in which an LLM-based task processing system identifies a region of interest in a target document entered by a user into the system. At this time, the target document may be at least one of a standard specification document by steel grade, a manufacturer's order specification document, a new standard document, and an arbitrary specification document set for manufacturing. In addition, any document capable of achieving the purpose of the present invention may be included as a target document regardless of its form or author.

[0073] In addition, the aforementioned area of ​​interest is an area within the document that the user wishes to analyze in detail, and may include text areas, table areas, and image areas, and the area of ​​interest may be determined differently depending on the type of task desired by the user.

[0074] In one embodiment, the region of interest may be an area containing text, and the region of interest may be identified by an AI-based network model that recognizes text within a document. In another embodiment, the region of interest may be an area containing an image, and the region of interest may be identified by an AI-based network model that recognizes images within a document. As those skilled in the art are likely already familiar with AI-based network models that extract text or images from documents, a detailed explanation thereof will be omitted.

[0075] In one embodiment, the region of interest may be a region containing a table composed of multiple rows and columns, and the region of interest may be identified by an artificial intelligence-based network model that recognizes a table included in a target document. Since the process of identifying a table included in a target document has been described in detail with reference to FIG. 2, a detailed explanation of redundant content will be omitted.

[0076] In step S200, reference data corresponding to the region of interest may be generated based on the analysis results of the region of interest identified through step S100. At this time, the region of interest may include at least one of a text region, a table region, and an image region.

[0077] In one embodiment, the region of interest may be an area where text is displayed (i.e., a text area), and the first reference data corresponding to the region of interest (i.e., a text area) may be the text (e.g., a string) itself or in a summarized form. In this case, the first reference data may be generated using a separately trained network model (i.e., a model different from the network model used to identify the region of interest) or using a previously disclosed text extraction technique. Additionally, the first reference data may be stored in a text list of a vector database.

[0078] That is, text regions in the target document can be identified through the first network model, and based on the analysis results of the text regions performed through the second network model, reference data in which strings corresponding to the text regions are preprocessed can be generated and stored in the database.

[0079] In one embodiment, the region of interest may be an area where a table is displayed (i.e., a table area), and the second reference data corresponding to the region of interest (i.e., a table area) may include structural information of the table and information written in each cell of the table. In this case, the first reference data may be generated using a separately trained network model (i.e., a model different from the network model used to identify the region of interest) or using a previously disclosed text extraction technique. Additionally, the second reference data may be stored in a table list of a vector database.

[0080] In one embodiment, the reference data may further include HTML tag data corresponding to a table displayed in a table area. In this case, the HTML tag data may be generated when the table contains a plurality of merged cells or when the number of cells exceeds a threshold. According to the present embodiment, by restricting the generation of HTML tag data according to predefined user conditions, unnecessary system operations can be reduced and computing resources can be managed efficiently.

[0081] In another embodiment, the HTML tag data can always be generated and stored in a database regardless of the table structure. This is a method of preprocessing all table structures into HTML tags to prepare for cases where it is difficult to predict in advance the conditions of a prompt that the user will input in the future. According to this embodiment, it is possible to prevent situations where processing fails or additional re-analysis is required because reference data is not generated in time when performing a specific task, enable flexible responses to prompt conditions, and ensure the stability of real-time task processing.

[0082] That is, a table area can be identified in the target document through the first network model, and based on the analysis results of the table area performed through the second network model, reference data with a preprocessed table corresponding to the table area can be generated and stored in the database.

[0083] Meanwhile, since the process of generating reference data by extracting the structure of the table and the information stored in each cell in the table area has been described above with reference to FIGS. 1 and FIGS. 2, a detailed explanation of the overlapping content will be omitted.

[0084] Referring again to FIG. 4, at step S300, the LLM-based task processing system can generate target data by inputting a prompt for processing tasks related to reference data and target documents into the LLM (Large Language Model). Subsequently, at step S400, the LLM-based task processing system can process the task using the target data.

[0085] At this time, the prompt may include at least one of information regarding the name of the specification item to be extracted within the target document entered by the user, the output format of the specification item, and the output method of the specification item. Additionally, the prompt may be determined based on at least one of the type of the target document and the type of the task.

[0086] That is, in one embodiment, the prompt for processing a task related to a target document may have all the information on specification items to be extracted by the user for each task type and the output format defined in advance.

[0087] Additionally, in another embodiment, the prompt for processing tasks related to the target document may be automatically generated by the system based on information (e.g., type of target document) obtained during the analysis process of the target document (e.g., identification of region of interest and generation of reference data) and user task history information related to the target document.

[0088] Meanwhile, details regarding the example prompt input to the LLM and the output corresponding to the example prompt will be described later with reference to FIGS. 8 to 16.

[0089] Additionally, depending on the conditions of the task or prompt to be executed, at least one of the steps in the above step S400 may be omitted. For example, in the case of a relatively simple task, task processing may be completed solely by generating the target data. On the other hand, in the case of a relatively complex task, additional processing steps to process document-related tasks desired by the user based on the generated target data may be performed sequentially.

[0090] FIGS. 5 to 7C are illustrative diagrams for explaining some operations illustrated in FIG. 4. More specifically, FIGS. 5 to 7C are illustrative diagrams for explaining target data generated as a result of inputting a table area identified in a target document, reference data stored in a database, and a prompt into the LLM.

[0091] Meanwhile, although the reference data itself generated as a result of analysis of the table area identified in the target document is not illustrated in FIGS. 5 to 7C, the reference data may refer to data including the structure of the table and the contents of each cell of the table, and may be stored in at least one of CSV format or HTML tag format.

[0092] Referring to FIG. 5, FIG. 5 shows a table area (i.e., area of ​​interest, 5a) of a target document, a first LLM output result (5b) when the output format is Excel, and a second LLM output result (5c) when the output format is HTML. At this time, the LLM output result can be understood as being generated based on a prompt entered by a user and reference data stored in a database within the system.

[0093] First, in the case of the first LLM output result (5b), reference data may be generated in CSV format and stored in a database as an analysis result for the table area (5a), and may be generated as a result of inputting the reference data and a prompt defined in the output method (i.e., Excel) into the LLM.

[0094] Next, in the case of the second LLM output result (5c), reference data may first be generated in the form of HTML tags and stored in a database as an analysis result for the table area (5a), and may be generated as a result of inputting the reference data and a prompt defined in the output method (i.e., HTML) into the LLM.

[0095] Referring to FIGS. 6A to 6C, FIG. 6A illustrates a table area (i.e., area of ​​interest, 6a) of a target document, FIG. 6B illustrates a first LLM output result (6b) when the output format is Excel format, and FIG. 6C illustrates a second LLM output result (6c) when the output format is HTML format. At this time, the LLM output result can be understood as being generated by LLM based on a prompt entered by a user and reference data stored in a database within the system.

[0096] As described with reference to FIG. 5, the first LLM output result (6b) may be generated by storing reference data in CSV format and inputting it into the LLM along with a prompt defining the output format. Additionally, the LLM output result (6c) may be generated by storing reference data in HTML tag format and inputting it into the LLM along with a prompt defining the output format.

[0097] In particular, in the case of the second LLM output result (6c), the visual layout of the table can be precisely represented as an HTML tag structure. For example, merged cells within the table can be implemented using HTML tags such as , and the multi-level header structure can also be reflected exactly as the original table format by utilizing a hierarchical structure such as , , etc.

[0098] Furthermore, each cell of the table may include metadata containing information of the source document from which the data was extracted (e.g., document name, page number, table ID, etc.), and the metadata may be implemented to be displayed as a tooltip when a user hovers a mouse over the cell. In this case, the tooltip may display information such as, for example, "Source: Standard_A.pdf, p.12, Table 3" or "Reference sentence: The content of C is defined as 0.15~0.20%".

[0099] Accordingly, according to the present disclosure, not only can the user easily grasp the result table visually, but they can also directly verify where each value was extracted within the document, thereby ensuring both the reliability and traceability of the data.

[0100] FIGS. 7A to 7C illustrate examples in which LLM output results are generated based on reference data and prompts according to the same process as FIGS. 5 and FIGS. 6A to 6C. The content exemplified in FIGS. 7A to 7C does not differ substantially from the content exemplified in FIGS. 5 and FIGS. 6A to 6C in terms of technical configuration or processing process. Therefore, FIGS. 7A to 7C can be understood as examples of variations in how results are expressed according to the output format (e.g., HTML or CSV), and detailed explanations regarding redundant content will be omitted.

[0101] In summary, the LLM-based task processing system of the present disclosure can select and utilize appropriate reference data according to the output format, thereby flexibly responding to various types of user task processing requirements. Furthermore, it can achieve superior effects in terms of processing speed and accuracy compared to manual document analysis and output operations.

[0102] Hereinafter, with reference to FIGS. 8 to 16, the types of prompts input into the LLM and the task processing results according to the prompt inputs will be explained. To elaborate, FIGS. 8 to 16 are diagrams illustrating the task processing results performed in an LLM-based task processing system that has been actually developed.

[0103] Figure 8 is a diagram illustrating an example of performing a task to extract chemical composition information from a target document.

[0104] As described with reference to FIGS. 1 to 4, the prompt input to the LLM may include information such as the name of the specification item to be extracted within the target document, the output format of the specification item, and the output method of the specification item.

[0105] More specifically, as illustrated in FIG. 8, the first prompt (8a) may include instructions to output the "Chemical Composition" item in the format 'Composition(ingredient name): value(value)'. Additionally, the first prompt (8a) may include a condition that if 'Cr / Ni / Mo' appears merged in a single column, it is treated as a single item and output as is without separating it into 'Cr', 'Ni', and 'Mo'. Furthermore, the first prompt (8a) may include additional explanatory information related to the task, such as the definition of the item, synonyms, and units (e.g., wt%).

[0106] Referring to FIG. 8, a first output result (8c) generated by the LLM using a first prompt (8a) for processing a task related to a target document and reference data is illustrated. At this time, the first output result (8c) can be configured in the form of a table in which the item name and corresponding value for each component, such as 'C [%]', 'Si [%]', and 'Mn [%]', are organized in pairs, centering on the 'Chemical Composition' item. In particular, the 'Cr / Ni / Mo [%]' item is not separated into 'Cr', 'Ni', and 'Mo' according to the prompt conditions, but is output as a single item, thereby confirming that the output result accurately matches the user's intention and instructions.

[0107] Meanwhile, FIG. 8 illustrates the original target document that serves as the basis for the first output result (8c), and it can be seen that the original target document actually contains a table (8b) titled 'Chemical Composition', and specifies a merge item 'Cr / Ni / Mo [%]' and a value 'to suit hardenability'. In this way, it can be seen that the structure and content of the table (8b) included in the original target document are accurately reflected in the first output result (8c), thereby indicating that item identification from the target document and prompt-based task execution have been successfully performed.

[0108] Figures 9A and 9B illustrate an example of performing a task to extract chemical composition information including rowspans from a target document.

[0109] As exemplified in FIG. 9A, the second prompt (9a) may include a directive to return all numeric data within the merged row without omission when the 'rowspan=2' attribute is detected in the tag area within the HTML structure. In addition, the above may mean a condition that induces the LLM to accurately extract all values ​​while maintaining the merged structure without separating them by item, even when a specific item (e.g., 'Ni', 'Si', etc.) within the table is located in a row merged with two or more values.

[0110] Additionally, FIG. 9B illustrates a first prompt (9a) for processing a task related to a target document and a second output result (9c) generated by the LLM using reference data. More specifically, the second output result (9c) may be configured in a table format in which the item names for each component, such as 'Al', 'B', 'C', 'Cr', 'P', and 'S' included in the 'Chemical Composition' item, and their corresponding numerical values ​​are organized in pairs. In particular, it can be confirmed that even items that were merged into two or more rows in the original document, such as 'Ni', 'Si', and 'Ti', have all numerical values ​​accurately output without omission according to the table structure.

[0111] Meanwhile, the original target document may include a chemical composition table (9b) labeled 'Spec-501001_J', and the table may be composed of columns such as 'element', 'maximum content', and 'permissible deviation'. At this time, the 'Ni', 'Si', and 'Ti' items are actually displayed in a row-merge format (i.e., rowspan=2), and it can be confirmed that multiple corresponding numeric values ​​exist. That is, by confirming that the structure and numeric information of the table (9b) within the original target document are accurately reflected in the second output result (9c), it can be confirmed that the LLM-based task can be performed accurately and completely even on a target document containing the structure of a merged cell.

[0112] FIGS. 10 and FIGS. 11 are drawings illustrating an example of performing a task to extract "Cleanliness" related item information from a target document.

[0113] First, referring to FIG. 10, the third prompt (10a) may include content that instructs to recognize information corresponding to the "Series" item and, if the related table structure is included in HTML format, to output the entire table in its original form. More specifically, the third prompt (10a) may be defined to determine that the table is related to the "Cleanliness" item if keywords such as 'Thick Series', 'Thin Series', 'Fine Series' are included in the columns of the table, or if values ​​such as 'A', 'B', 'C', 'D', 'DS' exist in the cells. To elaborate, this may mean a condition that induces even a table that was not previously directly named "Cleanliness" to be accurately classified as the corresponding item (i.e., "Cleanliness") through a keyword-based identification process, and to convert the entire table structure into HTML for structured output.

[0114] Next, referring to FIG. 11, a table (10b) included in the original target document, a third output result (10c) generated by the LLM using reference data, and a link click result (10d) that can be verified by clicking a hyperlink included in the output result (10c) are shown together.

[0115] First, the table (10b) in the original target document is a “Microscopic Cleanliness” table included in the original target document labeled ‘UC1_502010_H’, which may include items such as ‘Inclusion Type’, ‘Fine Series’, ‘Thick Series’, etc., and may have numerical values ​​for each type of item organized.

[0116] Next, in the third output result (10c), a hyperlink "See Cleanliness" along with a reference document name may be displayed in an area adjacent to the "Cleanliness" item, and when the user clicks the hyperlink, the related table may be connected to a separate sheet (e.g., link click result (10d)) in which the HTML structure is implemented. At this time, the table is composed of column names such as 'Inclusion Type', 'Fine Series', 'Thick Series', and numeric values ​​for each inclusion type such as 'A', 'B', 'C', 'D', and 'DS', similar to the table (10b) of the original target document (i.e., 'UC1_502010_H'), and can be confirmed to have been created to include the same row and column structure and data content as the original document.

[0117] In summary, based on the third prompt (10a), the structure and data content of the table (10b) within the original target document are accurately reflected in the third output result (10c) and the link click result (10d), thereby confirming that the LLM-based task can extract the entire table according to keyword-based table identification conditions and generate a visually and technically consistent output result.

[0118] FIGS. 12, FIGS. 13A, and FIGS. 13B are drawings illustrating an example of performing a task to extract information related to "Delivery Conditions" from a target document.

[0119] First, referring to FIG. 12, the fourth prompt (12a) may include a condition instructing that information related to the "Delivery Conditions" item be extracted, that text information be output in the form of key-value pairs classified according to major keys such as 'Hardness' and 'Microstructure', and that table-shaped information be included in its entirety in an HTML structure under the "Delivery Advice Note" item. Additionally, the fourth prompt (12a) may be defined to capture a wide range of related information by including various synonyms such as 'State of Delivery' and 'Applied Condition'. Furthermore, configurations expressed in a table, such as "Delivery Advice Note", may be additionally defined so that they can be accurately converted into an HTML table structure and output.

[0120] FIG. 13A shows the original target document (13a), FIG. 13B shows the 4-1 output result (13b) generated in correspondence with text within the original target document and the 4-2 output result (13cd) generated in correspondence with a table within the original target document.

[0121] First, on the "Delivery Conditions" related page of the original target document (i.e., 'Spec-501001_J', 13a), explanatory text regarding the delivery conditions may be placed at the top, and a structured table including columns such as 'Characteristic', 'Delivery Advice Note', and 'Test Certificate' may be placed at the bottom. To elaborate, the table may be configured in such a way that the entry status of each item in the delivery document is indicated by an 'X' symbol.

[0122] Next, the 4-1 output result (13b) is information extracted from the text portion within the original target document (13a), and it can be seen that the reference document name and the main description of the delivery conditions are displayed in the form of key-value pairs in the area adjacent to the "Delivery Conditions" item. On the other hand, the 4-2 output result (13c) is accessible via a hyperlink called "See Delivery Advice Note," and may be a visual reproduction on a separate sheet created by converting the table at the bottom of the original target document (13a) into an HTML table structure. It can be seen that the 4-2 output result (13c) reflects the visual structure of the original data exactly, such as the composition of rows and columns, item names, and 'X' markings.

[0123] In summary, based on the fourth prompt (12a), it can be confirmed that the text information and table structure information within the original target document are accurately reflected in the fourth-1 output result (13b) and the fourth-2 output result (13c), respectively, so that the LLM-based task can distinguish and process the information according to its representation method and generate an output result structured in an appropriate form. In particular, its technical utility is emphasized in that it is designed to comprehensively extract all information related to "Delivery Conditions" without omission by dividing the information extraction items for text and tables.

[0124] FIGS. 14 and FIGS. 15 are drawings illustrating an example of performing a task to extract "Carbon Segregation" related information from a target document.

[0125] First, referring to FIG. 14, the fifth prompt (14a) may include a special instruction condition that applies only to the "Carbon Segregation" item. More specifically, the fifth prompt (14a) may include a condition instructing that if the terms "Carbon" and "Size" are present together in the column headings of the table, the table be separated into two separate tables for extraction. This condition can be understood as being designed to enable clearer and more efficient analysis by structuring information by item, even when multiple semantically distinct information items are included together within the same table. Additionally, the fifth prompt (14a) may include content instructing that each separated table be output in HTML format.

[0126] FIG. 15 shows a table (14b) within the original target document, a fifth output result (14c) generated based on the corresponding prompt, and a first link click result (14d) and a second link click result (14e) that can be verified when each hyperlink is clicked.

[0127] First, the table (14b) in the original target document (i.e., 'Spec-501001_J') may be composed of columns such as 'Nominal Carbon content', 'Maximum Carbon variation', 'Product section size', 'Drill size', and may be presented in a form where information related to carbon content and product dimensions is integrated into a single table.

[0128] Next, based on the fifth prompt (14a), a fifth output result (14c) can be automatically generated in an area adjacent to the "Carbon Segregation" item, containing two hyperlinks (e.g., "See carbon table", "See size table") along with a reference document name (i.e., original target document name). When the user clicks each of the two hyperlinks, the related table information can be reproduced and output in an HTML structure on a separate sheet.

[0129] More specifically, the result of clicking the first link (14d) includes only information related to 'Carbon', and may be provided in the form of a table organized on a separate sheet based on the columns "Nominal Carbon content" and "Maximum Carbon variation". Additionally, the result of clicking the second link (14e) includes only information related to 'Size', and may be provided in the form of a table organized on a separate sheet based on the columns "Product section size" and "Drill size".

[0130] In summary, based on the fifth prompt (14a), information of different item types contained in a single table (14b) within the original target document can be provided in the form of links separated by item through the fifth output result (14c). Additionally, through the link click results (14d, 14e), it can be confirmed that the LLM-based task processing system can distinguish and sort complex information by item and generate visually and technically consistent analysis results by accurately outputting a table that structures the information. In particular, since the output result and the link click result accurately reflect the item structure and data content of the original target document (14b), the LLM-based task processing system of the present disclosure can prevent interpretation errors caused by mixing of information and simultaneously improve the accuracy and efficiency of data utilization by item.

[0131] FIG. 16 is a diagram illustrating an example of performing a task to extract inspection ratio information included in the "Anti-Mix Control" item from a target document.

[0132] As illustrated in FIG. 16, the sixth prompt (16a) may include a condition instructing the LLM to simply summarize and output only information in the format "inspection: number%" without unnecessary additional explanation from the context. More specifically, the sixth prompt (16a) can be understood as containing a user intention to ensure consistency and readability of output information by limiting the result to a structure containing only the key "inspection" and a numeric value (e.g., 100%).

[0133] Meanwhile, the text (16b) within the original target document (i.e., 'Spec-501001_J') may include an explicit expression "100% inspection of finished bars" under the "Anti-Mix Control" item. To elaborate, this expression describes the inspection requirements for preventing mixing before shipment and may be text containing quantitative numerical information.

[0134] The sixth output result (16c) generated by the LLM using the reference data generated based on the original target document and the sixth prompt (16a) can be seen as a result of extracting only the reference document name and the ratio of 'inspection' in the area adjacent to the "Anti Mix Control" item, and is composed of a key-value pair structure in the format {"inspection": "100%"}. That is, an output result can be provided that exactly matches the format specified in the prompt condition and accurately reflects the numerical information of the original text.

[0135] FIG. 17 is a hardware configuration diagram of an LLM-based task processing system according to some embodiments of the present disclosure. The LLM-based task processing system (1000) illustrated in FIG. 17 may include one or more processors (1100), a system bus (1600), a communication interface (1200), a memory (1400) for loading a computer program (1500) executed by the processor (1100), and a storage (1300) for storing the computer program (1500).

[0136] The processor (1100) controls the overall operation of each component of the LLM-based task processing system (1000). The processor (1100) can perform operations on at least one application or program for executing methods / operations according to various embodiments of the present disclosure. Memory (1400) stores various data, instructions and / or information. Memory (1400) can load one or more computer programs (1500) from storage (1300) to execute methods / operations according to various embodiments of the present disclosure. A bus (1600) provides communication functions between components of the LLM-based task processing system (1000). A communication interface (1200) supports internet communication of the LLM-based task processing system (1000). Storage (1300) can store one or more computer programs (1500) non-temporarily. A computer program (1500) may include one or more instructions in which methods / operations according to various embodiments of the present disclosure are implemented. When the computer program (1500) is loaded into memory (1400), a processor (1100) may perform methods / operations according to various embodiments of the present disclosure by executing the one or more instructions.

[0137] In some embodiments, the LLM-based task processing system (1000) described with reference to FIG. 17 may be configured using one or more physical servers included in a server farm based on cloud technology such as a virtual machine. In this case, at least some of the components shown in FIG. 17, such as the processor (1100), memory (1400), and storage (1300), may be virtual hardware, and the communication interface (1200) may also be configured as a virtualized networking element such as a virtual switch.

[0138] Various embodiments of the present disclosure and effects according to those embodiments have been described with reference to FIGS. 1 to 17. The effects according to the technical concept of the present disclosure are not limited to those described above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below.

[0139] The technical concept of the present disclosure described above may be implemented as computer-readable code on a computer-readable medium. The computer program recorded on the computer-readable recording medium may be transmitted to another computing device via a network such as the Internet and installed on the other computing device, thereby being used on the other computing device.

[0140] Although operations are depicted in a specific order in the drawings, it should not be understood that the operations must necessarily be executed in the specific order depicted or in a sequential order, or that all depicted operations must be executed to obtain the desired result. In certain situations, multitasking and parallel processing may be advantageous. Although embodiments of the present disclosure have been described above with reference to the attached drawings, those skilled in the art will understand that the present invention may be practiced in other specific forms without altering the technical concept or essential features thereof. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. The scope of protection of the present invention shall be interpreted by the claims below, and all technical concepts within the equivalent scope shall be interpreted as being included within the scope of rights of the technical concept defined by the present disclosure.

Claims

Claim 1 A method performed by at least one computing device comprising: identifying a region of interest in a target document that includes a table composed of multiple rows and columns; generating reference data corresponding to the region of interest based on an analysis result of the region of interest, wherein the reference data includes structural information of the table and information written in each cell of the table, and further includes HTML tag data corresponding to the table, wherein the HTML tag data is generated when the table includes multiple merged cells or the number of cells is greater than or equal to a threshold value; generating target data by inputting a prompt for processing a task related to the reference data and the target document into a Large Language Model (LLM), wherein the prompt includes at least one of information regarding the name of a specification item to be extracted within the target document, the output format of the specification item, and the output method of the specification item, and is determined based on at least one of the type of the target document and the type of the task; and processing the task using the target data, wherein the LLM-based task processing method comprises Claim 2 A method for processing tasks based on LLM according to claim 1, wherein the target document comprises at least one of a standard specification document by steel grade, a manufacturer's order specification document, a new standard document, and an arbitrary specification document set for manufacturing. Claim 3 A method for processing an LLM-based task according to claim 1, wherein the task is to extract information regarding a first specification from the target document, and the target data is an HTML tag in which the structure and content of a table containing information regarding the first specification are stored. Claim 4 An LLM-based task processing method according to claim 1, wherein the target document includes a first target document and a second target document, and the task is processed by selecting only the specification item corresponding to the intersection among the specification items extracted from the first target document and the second target document, respectively. Claim 5 An LLM-based task processing method according to claim 1, wherein the target document includes a first target document and a second target document, and the task is processed by selecting all specification items formed by a union among specification items extracted from the first target document and the second target document, respectively. Claim 6 An LLM-based task processing method according to claim 1, wherein the target documents are a plurality of target documents, and the task is processed by selecting the first specification item from the document with the highest priority according to a predefined priority among the target documents when the first specification item exists in common among the plurality of target documents. Claim 7 delete Claim 8 delete Claim 9 delete Claim 10 delete

Images