Large language model driven structured data interaction extraction and transformation system and method thereof

Through a large language model-driven system and multi-round dialogue prompt strategy, combined with a visual interactive interface, the problem of multi-themed information extraction in complex unstructured text is solved, and efficient and accurate table information extraction and personalized needs are achieved, improving the improvement of the improvement of efficient and accurate table information extraction and personalized needs. The efficiency and interactivity of data analysis.

CN119003711AActive Publication Date: 2024-11-22BEIJING INST OF TECH

Patent Information

Application Number
CN202410894912.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-04
Publication Date
2024-11-22
Estimated Expiration
2044-07-04

Smart Images

  • Figure CN119003711A_ABST
    Figure CN119003711A_ABST
Patent Text Reader

Abstract

The invention relates to a structured data interaction extraction and transformation system and method based on large language model driving, the system comprises a multi-table generation module, a table iteration derivation module and a TableQuilts interaction module, the method comprises the following steps: step 1, a user inputs an unstructured text, and the system preprocesses the input text; and 2, segmenting the original text into sub-texts with different theme meanings according to the content characteristics of the original text, and extracting a multi-theme table by a system according to the content of the divided text. The method has the advantages that the multi-theme structured form information is efficiently and accurately extracted from the unstructured text, the extraction process is continuously optimized through interaction between the user and the large language model so as to meet the personalized requirements of the user, the system improves the information processing efficiency and accuracy, and meanwhile the user experience is improved. The method enhances the interactivity and flexibility in the data analysis process, is suitable for various application scenes, and has wide application prospects and commercial values.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction, and in particular to a structured data interactive extraction and transformation system and method driven by a large language model. Background Art

[0002] The field of artificial intelligence (AI) has witnessed the rapid development and widespread application of large language models (LLMs) in recent years. These models have significantly pushed the boundaries of AI technology by processing and understanding complex language data. From Google's Transformer network to OpenAI's GPT series, each step of technological progress has significantly improved the processing power and application breadth of language models. Large language models have now become the core of modern computer science and artificial intelligence research, not only changing the way people process language, but also expanding the scope of application of machine learning in many industries.

[0003] Large language models are trained using huge datasets, which usually include extensive text materials collected from the Internet. In this way, the model is able to learn the deep structure and complex semantic information of the language, so as to generate coherent and logical text, and perform effective reasoning and question answering. For example, through pre-training and fine-tuning methods, these models can be customized and optimized on specific tasks to adapt to various needs ranging from simple classification tasks to complex text generation and dialogue systems. The training process of large language models usually involves multi-level, large-scale neural network architectures that are able to capture rich language features and extract useful information from huge text corpora.

[0004] When analyzing data, unstructured text does not conform to a common format or structure compared to structured information, and many unstructured texts contain information on multiple topics. In many scenarios, it is necessary to explore interesting structured information from these unstructured texts. For example, when a text worker gets a news report with rich content information, sometimes it is necessary to extract one or more data tables that respect the facts of the original text and have clear themes from the information in the report so that readers can understand the text more quickly, and thus quickly obtain content information on different topics that news practitioners want to emphasize without having to read a large amount of text.

[0005] With the explosive growth of unstructured data, unstructured data has occupied an increasingly large proportion of the information content on the Internet. Among the large amount of data from different sources, much of it is in the form of unstructured text, such as social media comments, emails, etc. Since unstructured text has no fixed structure and is flexible in representation, it contains rich information. The amount and complexity of unstructured data are increasing, and traditional data analysis and processing methods have been unable to cope with the growing demand. However, these data usually do not have a fixed structure and are difficult to process and analyze with simple rules or models. Therefore, an efficient method is needed to extract structured information from these unstructured texts so that users can quickly and accurately obtain the required information.

[0006] In actual tasks, it is often necessary to combine the text content, continuously explore the characteristics of the information in the text, and then extract the required structured information (usually in the form of a table) from it. In fact, extracting structured tables from unstructured text is a challenging task. Existing methods use the capabilities of natural language processing models or large language models to achieve automatic conversion from text to tables. However, these methods do not take into account people's actual needs in the process of extracting tables. When it is necessary to extract tables based on needs, it is necessary to use the capabilities of manual extraction or conversational AI. For unfamiliar texts, manually extracting tables from text is a time-consuming task, and conversational AI also needs to design prompt words based on the text and table content. Therefore, when dealing with demand-oriented text-to-table information extraction tasks, existing methods encounter two challenges: the complexity of text content and the uncertainty of user needs.

[0007] With the gradual development of large language models, large language models not only show extraordinary performance in traditional information extraction tasks, but also show great application potential in general information extraction tasks driven by natural language. The performance of large language models in these information extraction tasks provides a solution for the structured information extraction of unstructured texts: for the complexity of text content, since large language models have strong generalization, they can extract information in zero-shot and few-shot scenarios, so they have the ability to extract general information on different types of texts. For the uncertainty of user needs, although large language models show the application potential of general information extraction, it is difficult to rely solely on the capabilities of large language models to meet the changing needs of humans. Therefore, information can be extracted by interacting with large language models, focusing on the role of humans in the information extraction process. In the process of interacting with large language models, attention should be paid to human feedback information and the construction of appropriate prompt word strategies for model reasoning or model fine-tuning, so as to achieve demand-oriented information extraction.

[0008] The existing text-to-table conversion methods have the following technical problems:

[0009] Complexity of text content: The source text content is complex and diverse, and the content and potential structure of different texts vary greatly. Complex texts may contain information on multiple topics, and a single structured table is difficult to fully express this multi-topic information. Analyzing and processing these texts is a time-consuming and complex process.

[0010] Uncertainty of user needs: The characteristics of unstructured text data are not easy to be discovered intuitively, and user needs are diverse and changeable. It is difficult for automated methods to accurately extract tabular information that meets user specific needs. During the data exploration process, users may constantly adjust and discover new information needs, so flexible and interactive methods are needed to meet these changing needs.

[0011] The present invention aims to solve the above technical problems, specifically including:

[0012] Efficient text information extraction: Provides a system based on a large language model that can efficiently extract multi-topic structured tabular information from complex unstructured text. The system uses the powerful natural language processing capabilities of the large language model to achieve universal information extraction for different types of text.

[0013] Interactivity that meets user needs: Design a human-computer collaborative table extraction and exploration system that continuously optimizes and adjusts the extraction strategy through the interaction between humans and large language models, combined with user feedback and needs, to generate table information that meets the user's personalized needs.

[0014] Multi-round dialogue prompt strategy: Develop a multi-round chain prompt word strategy to guide the large language model to gradually extract important information from the text through multi-round dialogues and generate multi-topic structured tables to ensure the accuracy and comprehensiveness of the table information.

[0015] Visual Interaction Interface: Provides an intuitive visual interaction interface through which users can enter or upload text and interact with the generated tables to understand the text content more quickly, discover new information needs, and generate new tables. Summary of the invention

[0016] In order to solve the defects in the prior art, the present invention provides a structured data interactive extraction and transformation system and method driven by a large language model.

[0017] The present invention provides the structured data interactive extraction and transformation system driven by a large language model, comprising:

[0018] Multi-table generation module, table iteration derivation module, TableQuilts interaction module; among them, the multi-table generation module first splits the original text, and then extracts the initial generation of multi-topic tables based on the split text; the table iteration derivation module supports users to interactively iteratively derive new tables based on the initial table information and their own needs; the TableQuilts interaction module implements the interactive function when deriving tables, and the TableQuilts interaction module can also reflect the user's historical operations and help users intuitively understand the derivation relationship between tables.

[0019] The present invention further provides the structured data interactive extraction and transformation method driven by a large language model, comprising:

[0020] Step 1. The user inputs unstructured text, and the system preprocesses the input text;

[0021] Step 2. Segment the original text into sub-texts with different subject meanings according to the content characteristics of the original text. The system will extract a multi-subject table based on the content of the divided text;

[0022] Step 3. The system visualizes the initial table and verifies the table content, marks the table with content quality problems in the TableQuilts area, and prompts the user with the relationship between the initial table in an interactive form;

[0023] Step 4. The user selects a derivation mode according to his or her needs, and the system derives a new generation of tables based on the selected table and the derivation mode;

[0024] Step 5. The user implements interactive operations in the table derivation stage in the TableQuilts area, and the process waiting function set by the system manages the orderly progress of the table extraction process of unstructured text;

[0025] Step 6. The system has designed a complete error handling mechanism. When a network error or system abnormality occurs, the system records the current operation status and prompts the user to restore the operation.

[0026] Furthermore, in step 1, the user inputs unstructured text, and the system preprocesses the input text, specifically including:

[0027] Users input or upload original unstructured text through the system interface. The system supports users to manually input unstructured text including: text, pasted text and uploading local text files. The input text, pasted text and uploaded local text files will be automatically parsed and displayed in the text input area. The purpose of this process is to enable the system to receive texts in various formats and sources to ensure the applicability and flexibility of the system. The system pre-processes the input text and uploaded local text files to ensure the concise and standardized content.

[0028] Furthermore, in step 2, the original text is divided into sub-texts with different subject meanings according to the content characteristics of the original text. The system will extract a multi-topic table according to the content of the divided text, specifically including:

[0029] After the user displays the original text information in the input area of ​​the system, the prompt word of the multi-table generation module is called. The prompt word is divided into two stages: the first stage gathers the text of different subject contents into different sub-texts according to the content characteristics of the original text, and separates these sub-texts with different subject meanings with separators; the second stage generates a multi-subject table based on the text divided in the previous stage, in which:

[0030] The prompt words in the first stage follow the prompt words of the divide part in the divide-and-generate prompt words. According to the content characteristics of the original text, it is divided into sub-texts with different themes. The original long text is divided into several parts with different themes by separators.

[0031] In the second stage, the system will extract multi-topic tables based on the content of the divided text. The divide-and-generate prompt word method of generating multiple tables is to completely split each sub-text, and then use the generate prompt word for each sub-text to extract table information containing the title meaning from each sub-text. However, this method of generating tables is to completely split different sub-texts. When generating tables, it is difficult to consider the relationship between tables, resulting in large differences in the formats of tables with similar topics. In addition, tables have high requirements for the degree of data structuring, and some information that is difficult to express in tables will be discarded, resulting in obvious information loss.

[0032] In order to consider the association between sub-texts and reduce information loss, the system is designed with prompt words for multi-table generation. The prompt words include: task description, given segmented text, clear output form, and precautions. The prompt word task description first clarifies the task requirements, and then takes the entire segmented text as input and generates multi-topic tables based on the segmented text, rather than generating tables for each sub-text independently. This is conducive to ensuring a unified table generation style when generating table structures and extracting table content, and can generate tables with similar structures and expressions for similar sub-texts.

[0033] In order to ensure that the output form of the large language model is aligned with the target, the prompt word template clearly defines the output form and gives an example of the target format. The information of each table includes three parts: table subject, table header information, and table body information. In addition, the generate prompt word guides the large language model to fully consider the difficult-to-structure information in the text when generating the table, and add information supplementary columns to the table to record the precautions of this information, thereby reducing information loss. By using the above two-stage prompt words, users can extract the initial multi-topic table from the original text. Based on these initial formats, users can obtain a preliminary understanding of the original text and clarify what topics the original text data roughly contains. This helps users to conduct subsequent data exploration based on the content characteristics of the original text and the derived initial formats.

[0034] Furthermore, in step 3, the system visualizes the initial table and verifies the table content, marks the table with content quality problems in the TableQuilts area, and prompts the user with the relationship between the initial tables in an interactive form, including:

[0035] The system extracts the first generation of multi-theme tables from the text in the text input area. Users use this system to iteratively derive new tables from the original multi-theme tables. As the number of derived tables increases, the aesthetics and convenience of table reading in the table list area need to be guaranteed. For this reason, the table list area of ​​the system is designed in terms of table reading. The table list area supports users to scroll up and down to view all tables, and also supports scrolling left and right to view the complete information details of the table;

[0036] like Figure 1 As shown in the figure, the TableQuilts diagram in the TableQuilts interaction module is the main visualization form of the system. The TableQuilts diagram is composed of different rectangular blocks of color. The visualization form of the TableQuilts diagram in the system is displayed in the TableQuilts area. The construction process of the TableQuilts diagram is as follows: Figure 2As shown, the information containing all table attributes is stored in the form of json. By parsing the derived relationship between the tables in the json data and considering the position layout relationship of the target graphic elements, the json data is converted into a two-dimensional matrix form that is easy to render the TableQuilts diagram. Each element of the two-dimensional matrix corresponds to a rectangular element of the TableQuilts diagram. Finally, the two-dimensional matrix is ​​rendered into a graphical form. By adding interactive functions to this graphical form, the visual form of the TableQuilts diagram is obtained.

[0037] When passing Figure 1 After the multi-table generation module shown in the figure obtains the initial grid, a rectangular element representing the initial grid will appear in the TableQuilts area. In order to provide users with operation suggestions during interaction, elements describing table characteristics and table relationships are mapped on TableQuilts. The system has designed a table content inspection function. When rendering the TableQuilts diagram, all tables will be inspected one by one. Tables with complex cells and missing values ​​can be inspected and marked in the TableQuilts area.

[0038] In order to help users understand the relationship between the initial cells more quickly, the system calculates the similarity between the initial cells, and uses right-angle arrows to associate the tables with similar relationships. When the mouse hovers over the gray rectangle corresponding to the initial cell, a right-angle arrow will point from the current rectangle to the remaining rectangles with similar relationships. The mapping of the initial cell characteristics and the table relationship can help users clarify their next operation more quickly in the initial stage. For example, users can derive a new generation of tables based on the tables they are interested in and the tables that are related to them, or derive a subject table that they are interested in but has quality issues.

[0039] Furthermore, in step 4, the user selects a derivation mode according to his / her needs, and the system derives a new generation of tables according to the selected table and the derivation mode, including:

[0040] The user-selected derivation modes include random exploration, restricted exploration, and deterministic modification to meet the table derivation requirements with different levels of clarity, including:

[0041] Random exploration is suitable for the initial stage of exploration. At this time, users do not have a deep understanding of the initial table information. Users read the table's subject description information and select a single table of their interest or multiple related tables to derive a new generation of tables.

[0042] Restrictive exploration focuses on the subjective needs of users. Users use natural language to propose the information points they are concerned about or personalized derived needs. At this time, when deriving a single table or multiple tables, the user's needs will be fully considered, and a series of multi-topic tables that meet the user's needs will be derived. The user's needs do not constitute a strong constraint on the form of the target table, that is, the subject of the derived new table is not unique. In this mode, the large model will take into account the balance between exploration and demand constraints;

[0043] Deterministic modification means that the user has a clear modification requirement for a certain table. The user only needs to put forward clear modification suggestions. The system will modify the table according to the corresponding prompt words based on the content of the modification suggestion and display the modified table in the table derived list. The system will adjust the table content and structure based on the user's specific modification requirements to generate a table that meets the user's expectations.

[0044] The system supports flexible switching of multiple derivation modes. Users can choose the most appropriate derivation method according to actual needs and continuously optimize the table content. The flexibility of the system ensures that users can adjust strategies in time according to changes in needs during the process of exploring and deriving tables. The derived tables are displayed on the interface, and users view and manage the derived results through the interface. The system provides screening and filtering functions to help users efficiently manage and compare different table information. Users can filter out the most suitable tables according to specific needs. The system supports table version management. Users can view and trace back different versions of tables to ensure data integrity and consistency. The system also provides table export and sharing functions. Users can export table results into multiple formats for easy use in other applications or sharing with others.

[0045] Furthermore, in step 5, the user implements interactive operations in the table derivation stage in the TableQuilts area, and the process waiting function set by the system manages the orderly progress of the table extraction process of the unstructured text, including:

[0046] User interaction during the table derivation phase is done through Figure 1The TableQuilts interaction module in the TableQuilts area is implemented. The numbered rectangles in the TableQuilts area correspond to the tables in the table overview list. The numbers in the rectangles represent the numbers of the tables represented by the rectangles. The numbers of the tables correspond to the labels of the topics in the table overview list. Each column represents a derivation relationship, that is, the child cells are obtained from that or those parent cells. For each row, it is understood that each table in the table overview list participates in those derivation links. The gray rectangles arranged vertically in a column represent tables of the same generation. The multiple tables obtained by the multi-table generation module are primary cells, and the tables derived from the primary cells are secondary cells. And so on. For each generation of tables, the left area describes how the table is obtained, and the right area describes how these tables derive new tables.

[0047] In view of the fact that the system's workflow uses chained prompts, the order of the entire process needs to be kept correct during operation; in order to prevent the user's uncertain behavior or network factors from affecting the process sequence, the system has set up a process waiting function. The system has set up a stop waiting function for the multi-topic table generation module and the table iteration derivation module in the table extraction process of unstructured text. That is, for the iterative table exploration module, when the table starts to be derived, the derivation process will be marked as a working state. When in the working state, the user will be prompted to wait if he repeatedly clicks the derive button to avoid repeated or incorrect operations by the user. When a network error occurs or the derived sub-table table is obtained, the working state of the table iteration derivation module will be released, and the stop waiting function of the multi-table generation module The principle is similar. When the system generates the first generation of multi-topic tables, the multi-table generation module will be set to a working state. The working states of the multi-topic table generation module and the table iteration derivation module are equipped with a variety of work exit mechanisms to prevent user misoperation and network errors while ensuring system efficiency as much as possible. The system provides operation prompt functions, including repeated click prompts, prompts for unselected tables, and prompts for uninputted requirements. When the user does not select a derived table, does not raise a natural language requirement, or repeatedly clicks a function button, the system will prompt the user to perform the correct operation through a pop-up window or graphic language to avoid the trouble and time loss caused by misoperation. The system provides detailed user guides and help documents, which users can check and learn at any time to improve the convenience of system use and user experience.

[0048] Furthermore, in step 6, the system has designed a complete error handling mechanism. When a network error or system abnormality occurs, the system automatically records the current operation status and prompts the user to perform operation recovery, including:

[0049] Users can continue unfinished operations through the recovery function provided by the system to ensure data security and operation continuity. The system supports multi-level error monitoring and logging. Administrators can quickly locate and solve problems by viewing logs to improve system stability and reliability. Through this mechanism, the system can respond quickly and resume normal operation when an abnormal situation occurs, reducing the risk of user operation interruption and data loss.

[0050] The superior technical effects of the present invention are:

[0051] 1. The large language model-driven structured data interactive extraction and transformation system and method described in the present invention realize the efficient and accurate extraction of multi-topic structured table information from unstructured text, and continuously optimize the extraction process through the interaction between the user and the large language model to meet the personalized needs of the user. While improving the efficiency and accuracy of information processing, the system enhances the interactivity and flexibility in the data analysis process. It is suitable for a variety of application scenarios, such as news reporting, financial statements, market analysis, etc., and has broad application prospects and commercial value.

[0052] 2. The large language model-driven structured data interactive extraction and transformation system and method described in the present invention can quickly identify and extract multi-topic information in text through a multi-round chain prompt word strategy. Compared with traditional table extraction methods of text data that rely on manual rules or machine learning models, the speed and efficiency of data extraction are significantly improved when processing complex unstructured texts. The system can iteratively generate multi-topic structured tables in a short period of time, eliminating the need for users to spend a lot of time manually processing and analyzing texts.

[0053] 3. The large language model driven structured data interactive extraction system and method described in the present invention utilizes the powerful natural language processing capabilities of the large language model to accurately understand the semantics and contextual information in the text, thereby extracting high-quality structured tables. Through the improved divide-and-generate prompt word strategy, the text is divided into multiple sub-texts by topic, and the corresponding tables are generated to ensure that the generated table content is consistent with the original text, minimizing information loss and misunderstanding. In addition, when generating the table, the system considers the association between the tables to ensure the uniformity of the table format and content.

[0054] 4. The large language model-driven structured data interactive extraction and transformation system and method described in the present invention provide a variety of table derivation modes, including random exploration, restricted exploration and deterministic modification, so that users can freely select and generate structured tables that meet personalized needs based on their own needs and preferences; the system dynamically adjusts and optimizes the table generation strategy through interaction with the user and in combination with the user's feedback and needs, to ensure that the generated table can accurately reflect the user's needs.

[0055] 5. The large language model-driven structured data interactive extraction and transformation system and method described in the present invention, and the designed TableQuilts interaction module, provide an intuitive visual interactive interface. Users can intuitively view and manage the relationship and content characteristics between the primary table and the derived table through the TableQuilts diagram; the system establishes a mapping relationship between the table row data and the original text, and the user can view the corresponding part in the original text by clicking on the table row data. This interactive method helps users quickly understand the relationship between the table content and the original text, thereby improving data analysis efficiency.

[0056] 6. The large language model-driven structured data interactive extraction and transformation system and method described in the present invention are equipped with a complete system prompt and control module to ensure the sequential operation of the system and the smoothness of the user experience; the system is equipped with a process waiting function to prevent user misoperation or network problems from affecting the normal operation of the system; the system provides a variety of operation prompt functions, including repeated click prompts, unselected table prompts, uninput requirement prompts, etc., to ensure that users can use the system correctly and avoid operational errors; in addition, the system is also designed with an error handling and recovery mechanism. When a network error or abnormal operation occurs, the system can identify the current operation status and prompt the user to restore the operation to ensure the continuity of the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of the process of the large language model driven structured data interactive extraction and transformation method of the present invention;

[0058] Figure 2 It is a schematic diagram of the construction process of TableQuilts, the main visualization form of the structured data interactive extraction and transformation system driven by the large language model of the present invention. Implementation

[0060] The following is attached to the instruction manual Figure 1-2 And specific implementation methods, the technical solutions of the large language model-driven structured data interactive extraction and transformation system and method described in the present invention are further described in detail.

[0061] like Figure 1 As shown, the present invention provides the structured data interactive extraction and transformation system driven by a large language model, comprising:

[0062] Multi-table generation module, table iteration derivation module, TableQuilts interaction module. The multi-table generation module first splits the original text, and then extracts the initial multi-topic table based on the split text. Among them, the table iteration derivation module supports users to interactively iteratively derive new tables based on the information of the initial table and their own needs. The TableQuilts interaction module implements the main interactive functions when deriving tables. TableQuilts can also reflect the user's historical operations and help users intuitively understand the derivation relationship between tables. In the entire data conversion process, it not only utilizes the capabilities of the large language model, but also allows users to participate in decision-making in steps such as derivation mode selection and interaction with visualization forms.

[0063] The present invention further provides the structured data interactive extraction and transformation method driven by a large language model, comprising:

[0064] Step 1. The user inputs unstructured text, and the system preprocesses the input text: specifically,

[0065] Users input or upload original unstructured text through the system interface. The system supports users to manually input unstructured text including text, paste text and upload local text files. The input text, paste text and upload local text files will be automatically parsed and displayed in the text input area, so that the system can receive texts in various formats and sources to ensure the applicability and flexibility of the system. The system pre-processes the input text and uploaded local text files to ensure the conciseness and standardization of the content;

[0066] Step 2. Segment the original text into sub-texts with different subject meanings according to the content characteristics of the original text. The system will extract a multi-topic table based on the content of the divided text, including:

[0067] After the user displays the original text information in the input area of ​​the system, the prompt word of the multi-table generation module is called. The prompt word is divided into two stages: the first stage gathers the text of different subject contents into different sub-texts according to the content characteristics of the original text, and separates these sub-texts with different subject meanings with separators; the second stage generates a multi-subject table based on the text divided in the previous stage; among them:

[0068] The prompt words in the first stage follow the prompt words of the divide part in the divide-and-generate prompt words. According to the content characteristics of the original text, it is divided into sub-texts with different themes. The original long text is divided into multiple parts with different themes by separators.

[0069] In the second stage, the system will extract multi-topic tables based on the content of the divided text. The divide-and-generate prompt word method of generating multiple tables is to completely split each sub-text, and then use the generate prompt word for each sub-text to extract table information containing the title meaning from each sub-text. However, this method of generating tables completely splits different sub-texts, and it is difficult to consider the relationship between tables when generating tables, resulting in large differences in the formats of tables with similar topics. In addition, since tables have high requirements for the degree of data structuring, some information that is difficult to express in tables will be discarded, which will cause obvious information loss.

[0070] In order to consider the association between sub-texts and reduce information loss, the system is designed with prompt words for multi-table generation. Specifically, this prompt word includes: task description, given segmented text, clear output form, and precautions. The prompt word task description first clarifies the task requirements, and then takes the entire segmented text as input and generates multi-topic tables based on the segmented text, rather than generating tables for each sub-text independently. This is conducive to ensuring a unified table generation style when generating table structures and extracting table content, that is, for similar sub-texts, tables with similar structures and expressions will be generated;

[0071] In order to ensure that the output form of the large language model is aligned with the target, the output form is clearly defined in the prompt word template, and an example of the target format is given. The information of each table includes the table subject, table header information, and table body information. In addition, the generate prompt word guides the large language model to consider the difficult-to-structure information in the text when generating the table, and add information supplementary columns to the table to record this information, thereby reducing information loss. By using the above two-stage prompt words, users can extract the initial multi-topic table from the original text. Based on the multi-topic table, users can obtain a preliminary understanding of the original text and clarify which topics the original text data roughly contains. This helps users to conduct subsequent data exploration based on the content characteristics of the original text and the derived initial generation format;

[0072] Step 3. The system visualizes the initial table and verifies the table content. It marks the table with content quality problems in the TableQuilts area and uses an interactive form to prompt the user about the relationship between the initial tables, including:

[0073] The system extracts the first generation of multi-theme tables from the text in the text input area. Users use this system to iteratively derive new tables from the original multi-theme tables. As the number of derived tables increases, the aesthetics and convenience of table reading in the table list area need to be guaranteed. For this reason, the table list area of ​​the system is designed in terms of table reading. The table list area supports users to scroll up and down to view all tables, and also supports scrolling left and right to view the complete information details of the table;

[0074] like Figure 1 As shown in the figure, the TableQuilts diagram in the TableQuilts interaction module is the main visualization form of the system. The TableQuilts diagram is composed of different rectangular blocks of color. The visualization form of the TableQuilts diagram in the system is displayed in the TableQuilts area. The construction process of the TableQuilts diagram is as follows: Figure 2 As shown, the information containing all table attributes is stored in the form of json. By parsing the derived relationship between the tables in the json data and considering the position layout relationship of the target graphic elements, the json data is converted into a two-dimensional matrix form that is easier to render the TableQuilts diagram. Each element of the two-dimensional matrix corresponds to a rectangular element of the TableQuilts diagram. The two-dimensional matrix is ​​rendered in a graphical form. Adding interactive functions to this graphical form gives the visualization form of the TableQuilts diagram.

[0075] When passing Figure 1 After the multi-table generation module shown in the figure obtains the initial grid, a rectangular element representing the initial grid will appear in the TableQuilts area. In order to provide users with operation suggestions during interaction, elements describing table characteristics and table relationships are mapped on TableQuilts. The system has designed a table content inspection function. When rendering the TableQuilts diagram, all tables will be inspected one by one. Tables with complex cells and missing values ​​can be inspected and marked in the TableQuilts area.

[0076] In order to help users understand the relationship between the initial tables more quickly, the system calculates the similarity between the initial tables and uses right-angle arrows to associate the tables with similar relationships. When the mouse hovers over the gray rectangle corresponding to the initial table, a right-angle arrow will point from the current rectangle to the other rectangles with similar relationships. The mapping of the initial table characteristics and the table relationship helps users to more quickly clarify their next operation in the initial stage. For example, users can derive a new generation of tables based on the tables they are interested in and the tables that are associated with them, or derive a subject table that they are interested in but has quality problems.

[0077] Step 4. The user selects a derivation mode according to their needs. The system derives a new generation of tables based on the selected table and the derivation mode, including:

[0078] The user-selected derivation modes include random exploration, restricted exploration, and deterministic modification to meet the table derivation requirements with different levels of clarity, including:

[0079] Random exploration is suitable for the initial stage of exploration. At this time, users do not have a deep understanding of the initial table information. Users can read the table's subject description information and select a single table of their interest or multiple related tables to derive a new generation of tables.

[0080] Restricted exploration focuses on the subjective needs of users. Users use natural language to propose the information points they are concerned about or personalized derived needs. At this time, when deriving a single table or multiple tables, the user's needs will be fully considered to derive a series of multi-topic tables that meet the user's needs. The user's needs do not constitute a strong constraint on the form of the target table, that is, the subject of the derived new table is not unique. In this mode, the large model will take into account the balance between exploration and demand constraints;

[0081] Deterministic modification means that the user has a clear modification requirement for a certain table. The user only needs to put forward clear modification suggestions. The system will modify the table according to the corresponding prompt words based on the content of the modification suggestion and display the modified table in the table derived list. The system will adjust the table content and structure based on the user's specific modification requirements to generate a table that meets the user's expectations.

[0082] The system supports flexible switching of multiple derivation modes. Users can choose the most appropriate derivation method according to actual needs and continuously optimize the table content. The flexibility of the system ensures that users can adjust strategies in time according to changes in needs during the process of exploring and deriving tables. The derived tables are displayed on the interface, and users can view and manage the derived results through the interface. The system provides screening and filtering functions to help users efficiently manage and compare different table information. Users can select the most suitable table according to specific needs. The system supports table version management, and users can view and trace back different versions of tables to ensure data integrity and consistency. The system also provides table export and sharing functions. Users can export table results into multiple formats for easy use in other applications or sharing with others.

[0083] Step 5. The user implements interactive operations in the table derivation stage in the TableQuilts area. The process waiting function set by the system manages the orderly progress of the table extraction process of unstructured text, including:

[0084] User interaction during the table derivation phase is done through Figure 1The TableQuilts interaction module in the TableQuilts area is implemented. The numbered rectangles in the TableQuilts area correspond to the tables in the table overview list. The numbers in the rectangles represent the numbers of the tables represented by the rectangles. The numbers of the tables correspond to the labels of the topics in the table overview list. Each column represents a derivation relationship, that is, the child cells are obtained from that or those parent cells. For each row, it is understood that each table in the table overview list participates in those derivation links. The gray rectangles arranged vertically in a column represent tables of the same generation. The multiple tables obtained by the multi-table generation module are primary cells, and the tables derived from the primary cells are secondary cells. And so on. For each generation of tables, the left area describes how the table is obtained, and the right area describes how these tables derive new tables.

[0085] In view of the fact that the system's workflow uses chain prompts, the order of the entire process needs to be kept correct during operation; in order to prevent the user's uncertain behavior or network factors from affecting the process sequence, the system has set up a process waiting function. The system has set up a stop-wait function for the multi-topic table generation module and the table iteration derivation module in the table extraction process of unstructured text. For the iterative table exploration module, when the table starts to be derived, the derivation process will be marked as a working state. When in the working state, the user will be prompted to wait if he repeatedly clicks the derive button to avoid repeated or incorrect operations by the user. When a network error occurs or the derived sub-table table is obtained, the working state of the table iteration derivation module will be released. The stop-wait principle of the multi-table generation module It is similar. When the system generates the first generation of multi-theme tables, the multi-table generation module will be set to working status. The working status of the multi-theme table generation module and the table iteration derivation module sets a variety of work exit mechanisms to prevent users from operating incorrectly or ensure the system's operating efficiency when network errors occur. The system provides a variety of operation prompt functions, including repeated click prompts, unselected table prompts, and uninput requirement prompts. When the user does not select a derived table, does not raise a natural language requirement, or repeatedly clicks a function button, the system will prompt the user to perform the correct operation through a pop-up window or graphic language to avoid the trouble and time loss caused by incorrect operation. The system provides detailed user guides and help documents, which users can check and learn at any time to improve the convenience of system use and user experience.

[0086] Step 6. The system has designed a complete error handling mechanism. When a network error or system abnormality occurs, the system records the current operation status and prompts the user to restore the operation, including:

[0087] Users can continue unfinished operations through the recovery function provided by the system to ensure data security and operation continuity. The system supports multi-level error monitoring and logging. Administrators can quickly locate and solve problems by viewing logs to improve system stability and reliability. Through this mechanism, the system can respond quickly and resume normal operation when an abnormal situation occurs, reducing the risk of user operation interruption and data loss.

[0088] The present invention is not limited to the above-mentioned embodiments. Without departing from the essential content of the present invention, any deformation, improvement and substitution that can be thought of by those skilled in the art shall fall into the protection scope of the present invention.

Claims

1. A structured data interactive extraction and transformation system driven by a large language model, characterized in that: include: Multi-table generation module, table iteration derivation module, TableQuilts interaction module; the multi-table generation module first splits the original text, and then extracts the initial multi-topic table based on the split text; the table iteration derivation module supports users to interactively iterate and derive new tables based on the initial table information and their own needs; The TableQuilts interaction module implements the interactive function when the table is derived. The TableQuilts interaction module can also reflect the user's historical operations and help the user intuitively understand the derivation relationship between tables.

2. A method for using the structured data interactive extraction and transformation system driven by a large language model according to claim 1, comprising: Step 1. The user inputs unstructured text, and the system preprocesses the input text; Step 2. Segment the original text into sub-texts with different subject meanings according to the content characteristics of the original text. The system will extract a multi-subject table based on the content of the divided text; Step 3. The system visualizes the initial table and verifies the table content. It marks the table with content quality problems in the Table Quilts area and uses an interactive form to prompt the user of the relationship between the initial table. Step 4. The user selects a derivation mode according to his or her needs, and the system derives a new generation of tables based on the selected table and the derivation mode; Step 5. The user implements interactive operations in the table derivation stage in the TableQuilts area, and the process waiting function set by the system manages the orderly progress of the table extraction process of unstructured text; Step 6. The system has designed a complete error handling mechanism. When a network error or system abnormality occurs, the system records the current operation status and prompts the user to restore the operation.

3. According to the structured data interactive extraction and transformation method driven by a large language model as claimed in claim 2, in step 1, the user inputs unstructured text, and the system preprocesses the input text, specifically including: Users input or upload original unstructured text through the system interface. The system supports users to manually input unstructured text including: text, pasted text and uploading local text files. The input text, pasted text and uploaded local text files will be automatically parsed and displayed in the text input area. The purpose of this process is to enable the system to receive texts in various formats and sources to ensure the applicability and flexibility of the system. The system pre-processes the input text and uploaded local text files to ensure the concise and standardized content.

4. According to the structured data interactive extraction and transformation method based on large language model drive as claimed in claim 2, in step 2, the original text is divided into sub-texts with different subject meanings according to the content characteristics of the original text, and the system extracts a multi-topic table according to the content of the divided text, specifically including: After the user displays the original text information in the input area of ​​the system, the prompt word of the multi-table generation module is called. The prompt word is divided into two stages: the first stage aggregates the texts of different subject contents into different sub-texts according to the content characteristics of the original text, and uses separators to separate these sub-texts with different subject meanings; the second stage generates a multi-subject table based on the text divided in the previous stage.

5. According to the structured data interactive extraction and transformation method driven by a large language model as claimed in claim 2, in step 3, the system visualizes the initial table and verifies the content of the table, marks the table with content quality problems in the TableQuilts area, and prompts the user in an interactive form the relationship between the initial tables, specifically including: The system extracts the first generation of multi-topic tables from the text in the text input area. Users use this system to iteratively derive new tables from the original multi-topic tables. As the number of derived tables increases, the aesthetics and convenience of table reading in the table list area need to be guaranteed. For this purpose, the table list area of ​​the system is designed in terms of table reading. The table list area supports users to scroll up and down to view all tables and scroll left and right to view the complete information details of the table.

6. According to the structured data interactive extraction and transformation method based on large language model drive according to claim 2, in step 4, the user selects a derivation mode according to his own needs, and the system derives a new generation of tables according to the selected table and the derivation mode, specifically including: The user-selected derivation modes include random exploration, restricted exploration, and deterministic modification to meet the table derivation requirements with different levels of clarity, including: Random exploration is suitable for the initial stage of exploration. At this time, users do not have a deep understanding of the initial table information. Users read the table's subject description information and select a single table of their interest or multiple related tables to derive a new generation of tables. Restrictive exploration focuses on the subjective needs of users. Users use natural language to propose the information points they are concerned about or personalized derived needs. At this time, when deriving a single table or multiple tables, the user's needs will be fully considered, and a series of multi-topic tables that meet the user's needs will be derived. The user's needs do not constitute a strong constraint on the form of the target table, that is, the subject of the derived new table is not unique. In this mode, the large model will take into account the balance between exploration and demand constraints; Deterministic modification means that the user has a clear modification requirement for a certain table. The user only needs to put forward clear modification suggestions. The system will modify the table according to the corresponding prompt words based on the content of the modification suggestion and display the modified table in the table derived list. The system will adjust the table content and structure based on the user's specific modification requirements to generate a table that meets the user's expectations. The system supports flexible switching of multiple derivation modes. Users can choose the most appropriate derivation method according to actual needs and continuously optimize the table content. The flexibility of the system ensures that users can adjust strategies in time according to changes in needs during the process of exploring and deriving tables. The derived tables are displayed on the interface, and users view and manage the derived results through the interface. The system provides screening and filtering functions to help users efficiently manage and compare different table information. Users can filter out the most suitable tables according to specific needs. The system supports table version management. Users can view and trace back different versions of tables to ensure data integrity and consistency. The system also provides table export and sharing functions. Users can export table results into multiple formats for easy use in other applications or sharing with others.

7. According to the structured data interactive extraction and transformation method driven by a large language model as claimed in claim 2, in step 5, the user implements the interactive operation of the table derivation stage in the TableQuilts area, and the process waiting function set by the system manages the orderly progress of the table extraction process of the unstructured text, specifically including: The user's interactive operations in the table derivation stage are implemented through the TableQuilts interactive module. The numbered rectangles in the TableQuilts area correspond one-to-one to the tables in the table overview list. The numbers in the rectangles represent the numbers of the tables represented by the rectangles. The numbers of the tables correspond to the numbers of the topics in the table overview list. Each column represents a derivation relationship, that is, the child cells are obtained from that or those parent cells. For each row, it is understood that each table in the table overview list participates in those derivation links. The gray rectangles arranged vertically in a column represent tables of the same generation. The multiple tables obtained by the multi-table generation module are primary cells, and the tables derived from the primary cells are secondary cells. And so on. For each generation of tables, the left area describes how the table is obtained, and the right area describes how these tables derive new tables. The system's workflow uses chain prompts. During the operation, it is necessary to ensure that the order of the entire process remains correct. In order to prevent the user's uncertain behavior or network factors from affecting the process sequence, the system has set up a process waiting function. The system has set up a stop waiting function for the multi-topic table generation module and the table iteration derivation module in the table extraction process of unstructured text. That is, for the iterative table exploration module, when the table starts to be derived, the derivation process will be marked as a working state. When in the working state, the user will be prompted to wait if he repeatedly clicks the derive button to avoid repeated or incorrect operations by the user. When a network error occurs or the derived sub-table grid is obtained, the table iteration derivation module The working status will be released, and the stop waiting principle of the multi-table generation module is similar. When the system generates the first generation of multi-topic tables, the multi-table generation module will be set to the working status; the working status of the multi-topic table generation module and the table iteration derivation module have set up a variety of work exit mechanisms to prevent user misoperation and network errors. On the basis of ensuring system efficiency as much as possible, the system provides operation prompt functions, including repeated click prompts, unselected table prompts, and uninput requirement prompts. When the user does not select a derived table, does not make a natural language requirement, or repeatedly clicks the function button, the system will prompt the user to perform the correct operation through a pop-up window or graphic language to avoid the trouble and time loss caused by misoperation.

8. According to the structured data interactive extraction and transformation method based on large language model drive according to claim 2, in step 6, the system designs a complete error handling mechanism. When a network error or system abnormality occurs, the system automatically records the current operation status and prompts the user to restore the operation, including: Users can continue unfinished operations through the recovery function provided by the system to ensure data security and operation connectivity. The system supports multi-level error monitoring and logging. Administrators can quickly locate and solve problems by viewing logs, thereby improving system stability and reliability. When an abnormal situation occurs, the system can respond quickly and resume normal operation, reducing the risk of user operation interruption and data loss.

Citation Information

Patent Citations

  • Data interactive transformation and visual embedding method for complex table

    CN115391371A

  • Table analysis method, training method, equipment and computer readable storage medium

    CN116167381A

  • Text structuring method and system based on large language model

    CN117349275A

  • System and method for transformation of unstructured document tables into structured relational data tables

    US20210042309A1

Cited By

  • Function point duplicate checking method, device and equipment based on large language model application

    CN119271528A

  • Inference task execution method, electronic equipment, readable storage medium and program product

    CN119886360A

  • Artificial intelligence-based table generation method, system and equipment and storage medium

    CN120409440A