Structured information extraction method and system based on large language model

Abstract

The invention discloses a structured information extraction method and system based on a large language model, and the method comprises the following steps: carrying out the modular cue word design and adaptive loading, so as to guide the large language model to carry out the content generation according to the expected structure and logic; inputting text type identification and task routing to ensure that each type of information can be processed by adopting the most appropriate model prompt strategy; automatic segmentation and concurrent processing are carried out to ensure information integrity and processing efficiency; semantic completion and key information enhancement are carried out to carry out semantic completion on the entity information; performing structured format conversion and standard packaging to output a standardized format; and performing exception processing and full-process log recording to perform real-time tracking and backtracking on the state of each processing task. According to the method, the key information in the text can be automatically recognized, extracted and output in a structured mode, and the information processing efficiency and accuracy are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and particularly relates to a structured information extraction method and system based on a large language model. BACKGROUND

[0002] Currently, in the application scenarios of public opinion monitoring, government brief processing, major activity summary, etc., the information processing system usually relies on traditional natural language processing (NLP) technologies based on keyword matching, regular expressions, dictionary templates, etc. to extract key information from text. For example, by using named entity recognition (NER), syntax analysis, event extraction and other algorithm components, the time, place, participating subjects and core behaviors of the event and other key elements are extracted. This kind of system implementation is simple, low in deployment cost, and suitable for structured or language style consistent content processing tasks.

[0003] However, with the increasing diversification of information sources, the text data in real scenarios shows high complexity in expression, language style, structure level, etc. The traditional method has the following significant problems when processing such complex unstructured text: insufficient semantic understanding: unable to effectively understand the implicit logic or semantic association in the context, easily leading to information omission or extraction error; inconsistent extraction result granularity: for the same event, the system output may differ greatly under different expressions, making it difficult to standardize; poor adaptability: a set of rules or templates can only cover limited text patterns, and frequent adjustments are needed when facing frequently changing news briefs or emergency report scenarios.

[0004] In recent years, with the wide application of large language models based on the Transformer architecture (such as GPT, ChatGPT, LLAMA, DeepSeek, etc.), deep learning has made breakthrough progress in natural language understanding and generation. Large language models have strong semantic understanding, context analysis and language generation capabilities, and can be used for question answering, translation, abstract generation and information extraction, etc. tasks, especially in handling unstructured and complex semantic text. Existing research and engineering practice shows that a general large language model can be guided to extract the required information from the original text by designing a prompt. For example, through a prompt such as "Please extract the event location, time, and affected population from the following text", the model can automatically generate the target content. However, the practical application of such general models in information extraction tasks still faces many challenges, including: The output results are unstable and lack controllability: The output of large language models is highly dependent on the design of prompts. The generated content is affected by various factors such as model parameters, contextual semantics, and version differences. Even for the same input, the output may return results with different formats or content deviations. Lack of domain constraints and semantic adaptation: Current models are mostly general-purpose models and have not been trained for professional domains such as "policy briefings" and "event announcements". It is difficult to accurately capture key elements in fields such as administrative levels, professional terms, and names of statutory institutions. Weak structuring ability: The output is mostly natural language paragraphs without clear fields or tags. It is not easy to directly convert them into structured formats such as JSON and tables, and it cannot be connected to downstream automation systems.

[0005] In summary, although large language models have significant advantages in text understanding, their problems such as "uncontrollable output, lack of structure, and poor domain adaptability" severely limit their application in critical information extraction scenarios with high reliability and standardization requirements. They are especially unsuitable for structured content output needs for system integration and automatic analysis. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a structured information extraction method and system based on a large language model, which significantly improves the accuracy, stability, and system integration capabilities of large language models in specific domains, especially in personnel news briefings and texts related to major events (visits, meetings, etc.), effectively overcoming the limitations of general-purpose large models in existing technologies in terms of structured output and controllable generation.

[0007] This invention is achieved using the following technical solution: a structured information extraction method based on a large language model, comprising the following steps: Modular prompt word design and adaptive loading guide the large language model to generate content according to the expected structure and logic; Input text type recognition and task routing are implemented to ensure that each type of information is processed using the most appropriate model prompting strategy; Automatic segmentation and concurrent processing ensure information integrity and processing efficiency; Semantic completion and key information enhancement are used to semantically complete entity information; Structured format conversion and standard encapsulation to output standardized formats; Exception handling and full-process logging are implemented to track and backtrack the status of each processing task in real time.

[0008] Further, the modular prompt word design and adaptation loading specifically includes: for different types of information content, pre-design and configure a modular prompt template to guide the large language model to generate content according to the expected structure and logic. When the system receives the input text, it automatically matches and loads the corresponding prompt word according to its type, achieving precise adaptation.

[0009] Further, the input text type recognition and task routing specifically includes: identifying the business type of the text according to user input or preset rules, and routing it to the corresponding processing flow, so as to ensure that each type of information can be processed with the most suitable model prompt strategy.

[0010] Further, the automatic segmentation and concurrent processing specifically includes: for long texts exceeding the processing length limit of the large language model, automatically segmenting according to the maximum word limit threshold, and calling the large model to extract information in a concurrent manner, and finally integrating the results of each segment to ensure information integrity and processing efficiency.

[0011] Further, the semantic completion and key information enhancement specifically includes: introducing a mapping mechanism in the extraction result to complete the semantic completion of the extracted entity information.

[0012] Further, the mapping mechanism is a mapping mechanism for names and positions.

[0013] Further, the structured format conversion and standard encapsulation specifically includes: converting the extraction result from natural language form to unified structure field, encapsulating according to the field specifications used by different modules, and outputting in standardized JSON format for subsequent system integration and calling.

[0014] Further, the exception handling and full-process log recording specifically includes: built-in exception capture mechanism, exception interception for key steps of large model calling and result processing, and supporting log recording function, real-time tracking and backtracking of the state of each processing task, to improve the stability and maintenance ability of the system.

[0015] A structured information extraction system based on a large language model, for implementing the structured information extraction method based on a large language model described above, comprising: An input text preprocessing module for segmenting and processing user input text and identifying the type of the text according to the text features, while writing the corresponding prompt word template to provide context guidance for subsequent model calling; A prompt word loading and model calling module for loading the prompt word template matched with the identified text type, and calling the large language model interface, passing the processed text and prompt word template into the model, and returning the structured key information summary from the model; A key information extraction module is used to extract valuable information from the text and prompt word templates and output in a structured format. An exception handling and logging module is used to record task state management, database information, and error capture information. An information integration and structured output module is used to ensure that the returned data conforms to a standardized structure, facilitating subsequent system integration calls. An interface return and system integration module is used to return results through an API interface for further use by calling systems.

[0016] The present application has the following advantages: Compared with traditional natural language processing (NLP) methods, the present application has significant advantages. Traditional NLP methods usually rely on keyword matching, regular expressions, dictionary templates, and other techniques for information extraction. These methods often fail to provide accurate and stable results when faced with complex syntactic structures and diverse expression methods, and have limited semantic understanding depth. The present application, through deep semantic understanding and context association analysis of large language models, can more accurately identify key information in the text and automatically extract and structure the output, significantly improving the accuracy and efficiency of information processing.

[0017] Compared with traditional methods, the present application also has stronger adaptability and flexibility. When processing different types of text, the present application uses an adaptive type recognition mechanism and prompt word loading method to flexibly adjust the output strategy of the model according to the specific needs of the input text. This not only improves the accuracy of information extraction, but also enables the system to adapt to more types of text extraction tasks, such as personnel, strategy, finance, public opinion, meeting minutes, and other types of text, expanding its application scenarios and versatility.

[0018] In addition, the present application also has the ability to process long text, using a text segmentation mechanism to process long text that exceeds the model length limit, avoiding the truncation problem caused by input length limitations in traditional NLP methods. Furthermore, the semantic completion logic of the job level further improves the business integrity and accuracy of the information, ensuring the high availability and value of the extraction results in practical applications. Compared with traditional NLP methods, the present application not only reduces manual intervention, but also enables more efficient and accurate automated processing, with stronger technical advantages and broad application potential. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only show some of the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from the structures shown in the drawings without creative labor.

[0020] Figure 1 The data processing architecture diagram of the present application; Figure 2 The data processing timing diagram of the present application. DETAILED DESCRIPTION

[0021] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, not all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations.

[0022] It should be noted that: similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0023] The following will combine the drawings to specifically describe some embodiments of the present application. The following embodiments and features in the embodiments can be combined with each other without conflict.

[0024] Referring to Figure 1 , Figure 2 A structured information extraction method based on a large language model, comprising the following steps: Modular prompt design and adaptive loading: For different types of information content (such as personnel, strategy, meeting minutes, etc.), a set of modular prompt templates (Prompt) are designed and configured in advance to guide the large language model to generate content according to the expected structure and logic. When the system receives the input text, it will automatically match and load the corresponding prompt according to its type, achieving precise adaptation.

[0025] Input text type recognition and task routing: According to user input or preset rules, identify the business type of the text and route it to the corresponding processing flow, so as to ensure that each type of information can be processed with the most suitable model prompt strategy.

[0026] Automatic segmentation and concurrent processing mechanism: For long texts exceeding the processing length limit of large language models, the system automatically segments the text according to the maximum character limit threshold and calls the large model to extract information for each segment in a concurrent manner. Finally, the results of each segment are integrated to ensure information integrity and processing efficiency.

[0027] Semantic completion and key information enhancement: In the extraction results, the system introduces a mapping mechanism for names and positions to complete the semantic information of the extracted entities, such as automatically completing "Liu XX" to "Liu XX General Manager" to improve the accuracy and completeness of the business expression of the results.

[0028] Structured format conversion and standard packaging: The extraction results are converted from natural language form to unified structure fields (such as infoAbstract, keywords, etc.), packaged according to the field specifications used by different modules, and output in standardized JSON format for subsequent system integration and calling.

[0029] Exception handling and full-process logging: The system has built-in exception capture mechanisms to intercept exceptions in key steps such as large model calling and result processing, and is equipped with logging functions to track and backtrack the status of each processing task in real time, effectively improving the stability and maintenance ability of the system.

[0030] The present invention is suitable for key information extraction tasks in personnel, strategy, finance and capital finance, marketing, product, new technology, public opinion, and other news briefs, as well as important event text materials such as visits, visits, and meeting minutes. It can automatically identify, extract, and structure the key information in the text, improving information processing efficiency and accuracy.

[0031] The present invention also provides a structured information extraction system based on a large language model, which mainly includes an input text preprocessing module, a prompt word loading and model calling module, a key information extraction module, an exception handling and logging module, an information integration and structured output module, and an interface return and system integration module. The system is suitable for automatically extracting key information from multiple types of text and outputting structured results.

[0032] Input text preparation and preprocessing module: The user input text is a string containing multiple types of information, such as personnel, strategy, finance, and major events. After the system receives these texts, it performs preprocessing. The preprocessing work includes: segmenting long texts and identifying the type of text according to the text features, and writing corresponding prompt templates (Prompt) to provide context guidance for subsequent model calling.

[0033] Prompt loading and model invocation module, according to the text type identified in the first step, the system loads the prompt template matched with it. For example, personnel appointment, load extract_person_prompt prompt template. For major activities, load conference_prompt prompt template. The system then calls the large language model interface (this paper uses the locally deployed DeepSeek model), and transmits the processed text and prompt template into the model, which returns a structured key information summary.

[0034] Key information extraction module, after the large language model receives the text and prompt template, it extracts valuable information from it and outputs it in a structured format. Typical extraction tasks are as follows: Person information extraction: extract information such as name, position, unit, tenure, etc.

[0035] Output format example: { "infoAbstract": "Beijing, Zhang San, born in May 1975, served as xx in April 2023, previously served as xx." }。

[0036] Product information extraction: extract vehicle positioning, price, size, design highlights, technical parameters, etc.

[0037] Output format example: { "infoAbstract": "Recently, Tesla released the latest Model Y, positioning it as a mid-size SUV. The price starts at 350,000 yuan, and the vehicle size is 4900x1900x1650mm. The exterior design is simple, the interior design is mainly black, and it is equipped with the latest automatic driving function. In terms of power, it is equipped with a 350-horsepower electric motor." "keywords": ["Tesla", "Model Y", "SUV", "electric", "automatic driving"] }。

[0038] Major event information extraction (outbound / visiting / conference): the system determines the text as outbound, visiting or conference through keyword and structure recognition, and extracts elements such as time, location, characters and activity content to generate an information summary.

[0039] Outbound activity example: { "infoAbstract": "On XX day, XXX went to XX location to visit XXX unit and had a discussion with XXX leader (or both parties) on the topic of concern." }。

[0040] Visit Activity Example: { "infoAbstract": "On XX, XXX visited XX location and held discussions on relevant topics. XX attended the event." }。

[0041] Meeting Minutes Example: { "infoAbstract": "On XX, XXX held a meeting at XXX location. XXX attended the meeting." }。

[0042] In addition, the system automatically completes the names of the extracted results with their positions when processing major event texts, enhancing the semantic integrity.

[0043] Exception Handling and Logging Module: To ensure system stability and information traceability, the invention adds a comprehensive exception handling and logging mechanism during processing: Task Status Management: A unique task record is generated each time the call is made, and the status is noted (e.g., in progress, success, failure); Database Recording: Records input text, model invocation request, model return content, and structured results; Error Capture: If the model returns an exception or the structure extraction fails, the system will automatically record error information for user query and operation troubleshooting.

[0044] Information Integration and Structured Output Module: When the model returns key information, the system will further integrate the results to ensure that the returned data conforms to the standardized structure, facilitating subsequent system integration and invocation. The output of each module is uniformly packaged in JSON format to ensure information consistency and ease of use.

[0045] Output Example: { "status": 200, "message": "Success", "data": { "infoAbstract": [ "On April 8, 2025, discussions on A issues were held, and xx responsible person proposed a new economic reform scheme at the meeting." ], "keywords": ["A", "economic", "reform"] } }。

[0046] The interface returns the system integration module, finally, the structured output result is returned in the form of an API interface, for calling systems (such as government systems, knowledge management platforms, or public opinion analysis systems) for further use. The interface data format is uniform and the fields are standardized, supporting efficient integration and automated processing.

[0047] The application greatly improves the efficiency and accuracy of information extraction by preprocessing of input text, type recognition, prompt word loading, calling of a large language model, key information extraction, exception handling, and log recording. The method can be widely applied in the fields of personnel, strategy, finance, products, technology, etc. to report and visit, visit, conference minutes, and other major activities, realizing automatic extraction and structured processing of information.

[0048] For the foregoing embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the application is not limited by the order of the described actions, because according to the application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the application.

[0049] In the above embodiments, the basic principles and main features of the application and the advantages of the application are described. Those skilled in the art should understand that the application is not limited by the above embodiments, and the above embodiments and the description in the specification are only to illustrate the principles of the application. Any modification and change made by those skilled in the art without departing from the spirit and scope of the application should be within the protection scope of the claims of the application.

Claims

1. A method for structured information extraction based on a large language model, characterized in that, Comprising the following steps: Modular prompt word design and adaptive loading to guide large language models to generate content according to expected structure and logic; Input text type recognition and task routing to ensure that each type of information is processed using the most appropriate model prompt strategy; Automatic segmentation and concurrent processing to ensure information integrity and processing efficiency; Semantic completion and key information enhancement to complete entity information semantically; Structured format conversion and standard encapsulation to output standardized formats; Exception handling and full-process logging to track and backtrack the status of each processing task in real time. 2.The method of claim 1, wherein, The modular prompt word design and adaptive loading specifically includes: for different types of information content, pre-design and configure modular prompt templates to guide large language models to generate content according to expected structure and logic, when the system receives input text, automatically match and load the corresponding prompt words according to its type, achieving precise adaptation. 3.The method of claim 1, wherein, The input text type recognition and task routing specifically includes: identifying the business type of the text according to user input or preset rules, and routing it to the corresponding processing flow, so that each type of information can be processed using the most appropriate model prompt strategy. 4.The method of claim 1, wherein, The automatic segmentation and concurrent processing specifically includes: for long texts that exceed the processing length limit of large language models, automatically segment them by the maximum character limit threshold, and use concurrent mode to call large models to extract information segment by segment, and finally integrate the results of each segment to ensure information integrity and processing efficiency. 5.The method of claim 1, wherein, The semantic completion and key information enhancement specifically includes: introducing a mapping mechanism to complete the semantic of the extracted entity information in the extraction results. 6.The method of claim 5, wherein, The mapping mechanism is a mapping mechanism for names and positions.

7. The structured information extraction method based on a large language model of claim 1, wherein, The structured format conversion and standard encapsulation specifically includes: converting the extraction results from natural language form to unified structure fields, encapsulating according to the field specifications used by different modules, and outputting in standardized JSON format for subsequent system integration and calling. 8.The method of claim 1, wherein, The exception handling and full-process logging specifically includes: built-in exception capture mechanism, exception interception for key steps of large model calling and result processing, and supporting log recording function, real-time tracking and backtracking of the status of each processing task to improve the stability and maintenance ability of the system.

9. A large language model-based structured information extraction system for implementing the large language model-based structured information extraction method of any one of claims 1-8. Comprising: An input text preprocessing module for segmenting and processing user input text and identifying the type of the text according to its characteristics, while writing the corresponding prompt word template to provide context guidance for subsequent model calling; A prompt word loading and model calling module for loading the prompt word template matched with the identified text type, and calling the large language model interface, passing the processed text and prompt word template into the model, and returning the structured key information summary from the model; A key information extraction module for extracting valuable information from the text and prompt word template and outputting it in a structured format; An exception handling and logging module for recording task status management, database information, and error capture information; Information integration and structured output module, to ensure that the returned data conforms to the standardized structure, to facilitate subsequent system integration call; Interface return and system integration module, to return the results through the API interface form, for the calling system for further use.

Images