Technical document key information extraction method based on intelligent agent

Abstract

The invention provides an agent-based technical document key information extraction method. The method comprises the following steps: receiving text information input by a user; the intelligent agent calls a large language model to disassemble the extraction action to form an extraction sub-task of the extraction target; the large language model determines a tool to be called for each extraction sub-task according to the tool description, and a task arrangement sequence is formed; and the large language model calls the tool to be called to process the technical document information until all the extraction actions are executed. According to the method, the intelligent agent is constructed, the big language model is used as the brain of the intelligent agent, the extraction operation information input by the user is understood and subjected to task splitting, corresponding tools are autonomously thinking and called to complete key information extraction of the technical document information, and setting of a specific process is not needed; and the intelligent agent dynamically generates a task flow which most meets the user requirement and outputs a result, so that information extraction of any type of technical documents is met, and the flexibility and the expandability are greatly improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular, to a method, apparatus, storage medium, and electronic device for extracting key information from technical documents based on an agent. Background Art

[0002] With the continuous increase of global innovation activities, the number of various technical documents such as patent documents, journals, and scientific papers has increased exponentially, and the large amount of technical information contained in the documents has become increasingly important in aspects such as technology trend prediction, competitive intelligence analysis, and technology research and development. However, the professionalism, complexity, and diversity of technical documents make it impossible for users to quickly and accurately obtain the key information in the documents when reading technical documents. Therefore, extracting key information from technical documents has become a common way for users to quickly understand technical texts.

[0003] Traditional information extraction mostly uses rule-based and traditional NLP (Natural Language Processing) technologies to implement. The rule-based method requires sorting out different rules for different fields, while the NLP technology-based method requires a large amount of data annotation by professionals with expertise to assist model training and optimization. At the same time, the task process can only be carried out in the order established by developers, often unable to meet the personalized key information extraction needs of users, and both flexibility and scalability are at a relatively low level. Summary of the Invention

[0004] The purpose of the embodiments of the present disclosure is to provide a method, apparatus, storage medium, and electronic device for extracting key information from technical documents based on an agent, so as to solve the problem that the flexibility and scalability of the key information extraction method for technical documents in the prior art are both at a relatively low level.

[0005] Embodiments of the present disclosure adopt the following technical solutions: A method for extracting key information from technical documents based on an agent, comprising: The agent receives text information input by a user, and the text information at least includes: technical document information and extraction operation information, and the extraction operation information at least includes: at least one extraction target and an extraction action performed on the extraction target; The agent calls a large language model to disassemble the extraction action corresponding to each extraction target according to each extraction target, forming at least one extraction subtask of the extraction target; The large language model determines at least one tool to be called for each extraction subtask according to the tool description, and forms a call order between all the tools to be called according to the logical relationship between the extraction subtasks, so as to form a task choreography sequence; The large language model sequentially calls the tools to be called in the task choreography sequence to process the technical document information until the extraction actions for all the extraction targets are completed, forming an output result and presenting it to the user.

[0006] Embodiments of the present disclosure also provide an apparatus for extracting key information from technical documents based on an agent, comprising: An interaction module, configured to call an agent to receive text information input by a user, and the text information at least includes: technical document information and extraction operation information, and the extraction operation information at least includes: at least one extraction target and an extraction action performed on the extraction target; An operation module, configured to call a large language model to disassemble the extraction action corresponding to each extraction target according to each extraction target, forming at least one extraction subtask of the extraction target; The large language model determines at least one tool to be called for each extraction subtask according to the tool description; The large language model forms a call order between all the tools to be called according to the logical relationship between the extraction subtasks, so as to form a task choreography sequence; The large language model sequentially calls the tools to be called in the task choreography sequence to process the technical document information until the extraction actions for all the extraction targets are completed, forming an output result and presenting it to the user.

[0007] Embodiments of the present disclosure also provide a storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the steps of the above-mentioned method for extracting key information from technical documents based on an agent are implemented.

[0008] Embodiments of the present disclosure also provide an electronic device, at least including a memory and a processor, where the memory stores a computer program, characterized in that when the processor executes the computer program on the memory, the steps of the above-mentioned method for extracting key information from technical documents based on an agent are implemented.

[0009] The beneficial effects of the embodiments of the present disclosure are as follows: By constructing an agent and using a large language model as the brain of the agent, the extraction operation information input by the user is understood and the task is disassembled. Subsequently, the agent independently thinks and calls the corresponding tools to complete the extraction of key information from technical document information. Without having to execute according to a fixed process setting, the agent can dynamically generate the task process that best meets the user's needs and output the results. At the same time, it can meet the information extraction of any type of technical document, greatly improving flexibility and scalability. Description of the Drawings

[0010] In order to more clearly illustrate the technical solutions in one or more embodiments of this specification or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings described below are only some embodiments recorded in this specification. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0011] Figure 1 It is a flowchart of a method for extracting key information from technical documents based on an agent in the first embodiment of the present disclosure; Figure 2 It is a flowchart of a method for constructing an agent in the first embodiment of the present disclosure; Figure 3 It is a schematic structural diagram of an apparatus for extracting key information from technical documents based on an agent in the second embodiment of the present disclosure. Detailed Embodiments

[0012] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will clearly and completely describe the technical solutions in one or more embodiments of this specification in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only some embodiments of this specification, rather than all embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this document.

[0013] With the continuous increase of global innovation activities, the number of various technical documents such as patent documents, periodicals, and scientific papers has increased exponentially. The large amount of technical information contained in the documents has become increasingly important in aspects such as technology trend prediction, competitive intelligence analysis, and technology research and development. However, the professionalism, complexity, and diversity of technical documents make it impossible for users to quickly and accurately obtain the key information in the documents when reading technical documents. Therefore, extracting key information from technical documents has become a common way for users to quickly understand technical texts.

[0014] Traditional information extraction mostly uses rule-based and traditional NLP techniques to achieve. The rule-based approach requires sorting out different rules for different domains, while the NLP technique-based approach requires a large amount of data annotation by professionals with expertise to assist in model training and optimization. At the same time, the task process can only be carried out in the order established by developers, often unable to meet the user's personalized key information extraction needs, and both flexibility and scalability are at a relatively low level. In addition, for document content with complex language structures and diverse expressions such as patents and papers, the conventional rule-based and template-based methods have poor generalization ability and are difficult to cover comprehensively. When encountering expressions or language patterns not defined by the rules or templates, they cannot be processed, resulting in a decrease in the extraction accuracy; in some cases, the language of technical documents will change continuously with the development of time and technology, and the traditional information extraction methods cannot automatically adapt to these changes, requiring continuous maintenance and update of rules and models, resulting in increased costs; and the rule-based and model-based methods usually require in-depth participation of domain experts in construction, and separate rules and training models need to be designed for each technical field, resulting in a long development cycle, high costs, and it is difficult to migrate to other fields; moreover, the traditional methods also have deficiencies in multi-language support capabilities, and different task processing models need to be trained for different languages, with relatively large costs.

[0015] To solve the above problems, the first embodiment of the present disclosure provides an agent-based method for extracting key information from technical documents, using an Agent constructed based on the LangChain framework to achieve automatic understanding and processing of the user's extraction requirements. Figure 1 The flowchart of the agent-based method for extracting key information from technical documents described in this embodiment is shown. This method can be used as the actual processing logic of a question-and-answer system with an operable interface. The user inputs a question through the operable interface, and after the agent forms an answer, the result is displayed to the user through the operable interface. The main steps of this embodiment include steps S10 to S40: S10, the agent receives the text information input by the user.

[0016] The text information in this embodiment is the information input by the user in the operable interface through natural language, which at least includes technical document information and extraction operation information; among them, the technical document information refers to the technical text from which the user hopes to extract key information, that is, the text content that the user actually hopes to extract information from, or the identifier used to represent the technical text, such as the publication number or application number of a patent, the journal article number, the DOI (Digital Object Identifier) of a paper, etc. Through these identifiers, the content of the corresponding technical document can be obtained; and the extraction operation information is used to represent the specific operations that the user expects to perform on the technical text. In this embodiment, the extraction operation information at least includes: at least one extraction target and the extraction action performed on the extraction target. The extraction target refers to the specific type of key information in the technical document that the user expects to obtain, such as the technical problem, technical means, and technical effect of a patent, the research background and core issues, research methods and implementation paths, and research conclusions and value extensions of a journal paper, etc. The extraction action mainly refers to the extraction of key information. The user can also perform further action instructions such as warehousing action and charting action according to their own needs.

[0017] In some embodiments, the text information input by the user can be "Extract the technical problems and technical means of the patent with the publication number CN12345678A and save the extracted technical problem keywords to the database", or "Extract the content of the article with the article number 1003 - 2797(1997)02 - 0013 - 05 and present it in tabular form", or "Extract the technical problems of the following text {{{There are many reasons for poor lubrication, and many faults are accompanied by poor lubrication. The abnormal vibration caused by poor lubrication belongs to forced vibration of random excitation. The main excitation source is a large number of disordered random bump rubs. The response of the rolling element and raceway rub is an unordered random excitation frequency, which is a broadband "white noise" excitation. After poor lubrication occurs, it generally causes various derivative effects, such as slipping, raceway damage, cage jamming, etc. These derivative effects will affect the vibration spectrum.}}}" It should be noted that the above text information is only an example of some text information content that the user may input, and does not limit the content actually input by the user.

[0018] An intelligent agent (Agent) refers to a system that can autonomously sense the environment, make decisions, and execute actions. It has basic characteristics such as autonomy, interactivity, reactivity, and adaptability, and can independently complete tasks in a complex and changing environment. The intelligent agent in this embodiment is built based on LangChain and mainly includes a large language model (LLM, Large Language Model), tools, and short - term memory (Memory).

[0019] Specifically, the LLM is a large deep learning model pre-trained on a large amount of data that can generate responses to user queries and acts as the brain in an agent. It is typically used to understand user input, generate responses, and in some cases make decisions or provide suggestions. In an agent built using LangChain, the large language model can adjust the scheduling strategy in combination with the ReAct mechanism, that is, through the loop of Thought (thinking) → Action (execution) → Observation (observation) → Thought (thinking) update until the problem is solved or the task end condition is met. Tools are an important part of the agent built by LangChain. They enable the agent to perform actual operations, call external services (APIs or custom tools), and interact with the system, thus increasing the functionality and scope of application of the agent. By integrating different types of Tools, the agent can complete more complex tasks, such as data query, information retrieval, web scraping, and information processing. Short-term memory plays the role of context retention, personalized response, and state management. It allows the agent to maintain the context of the conversation in multiple interaction steps, so that the agent can remember the previous communication content when performing tasks and provide continuous and personalized responses. The combination of tools, short-term memory, and large language models enables LangChain to build highly intelligent and automated applications.

[0020] When actually building an agent, it can be carried out in combination with Figure 2 the process shown. First, build a tool library that includes at least a data acquisition tool and a key information extraction tool, which are used to perform the actions of acquiring technical texts and extracting key information respectively; then build a large language model and add the tool set to the skill list of the large language model to enable the large language model to have the capabilities of the above-built tool library; after building the large language model and the tool library, by using the create_react_agent() method of LangChain and taking the large language model and the tool library as parameters of the LangChain framework, an agent can be built.

[0021] Specifically, the data acquisition tool is used to obtain technical texts according to technical document information, parse and extract the text content corresponding to different chapters of the technical texts, and splice and assemble the text content of different chapters into at least one text dictionary according to preset rules according to the extraction target; among them, when the technical document information includes the technical text, the data acquisition tool directly obtains the technical text, and when the technical document information is an identifier used to represent the technical text, the data acquisition tool calls the data acquisition interface according to the identifier to obtain the technical text, such as obtaining patent texts from the patent database according to the patent publication number, obtaining journal article content from the journal database according to the journal article number, etc. When parsing and extracting the text content, different chapter content extraction can be carried out in combination with the content structure of different types of technical documents. For example, for patent documents, extraction can be carried out separately according to parts such as the claims, background art, invention content, specific embodiments, etc., and for journal papers, extraction can be carried out separately according to parts such as the abstract, introduction, research methods, results and discussion, etc. It should be noted that the preset rules of this embodiment can be configured in combination with the extraction target, and the content mainly corresponding to the extraction target is spliced to form a text dictionary. For example, when the extraction target is technical means and technical problems, the claims part and the specific embodiments part can be spliced as a text dictionary for realizing the extraction of technical means, and the background art part and the invention content part are spliced to form another text dictionary for extracting technical problems. However, the above preset rules are only for illustration, and the actual splicing method can be set by developers or autonomously set by the large language model according to the extraction target, and this embodiment does not make specific restrictions.

[0022] The key information extraction tool of this embodiment is mainly used to form task tags for each extraction target, which can be represented by a unique string; then the key information extraction tool calls the extraction model to extract key information from the text dictionary matching the extraction target, and encapsulates the extraction result and the task tag corresponding to the extraction target into an extraction result dictionary. The extraction model of this embodiment uses the Universal Information Extraction (UIE) as the base model, and fine-tunes the base model by using doccano to annotate multi-language classification data to obtain an extraction model for completing specific extraction targets. Moreover, this model integrates semantic information and can adaptively understand the continuously updated article content expression, effectively improving the intelligent agent's understanding ability of technical texts and enhancing the subsequent extraction accuracy. At the same time, the large language model has the ability of multi-language understanding. When training the UIE model, multi-language annotation data can be used for training to meet the user's key information extraction requirements for technical documents in different languages and solve the limitation of single model processing single language.

[0023] In some embodiments, according to different extraction targets, the key information extraction tools can also be subdivided according to specific extraction targets. For example, a technical problem extraction tool, a technical means extraction tool, and a technical effect extraction tool can be constructed respectively according to the technical problems, technical means, and technical effects of patent documents, so as to improve the extraction effect of the extraction tool on specific extraction targets. Correspondingly, when training the extraction tools of specific categories, the same UIE model can be used as the base model, but training is carried out using the annotation data with corresponding specific categories.

[0024] Furthermore, in order to improve the accuracy of the extraction results and meet the user's additional document content processing requirements, the tools in the tool library can also include at least one of a data preprocessing tool, a key information verification tool, a data storage tool, and a model optimization tool; among them, the data preprocessing tool is used to clean the data irrelevant to the technology in the technical text. Specifically, it can clean the HTML tags contained in the text dictionary by a mixed cleaning method of regular expressions and an HTML parsing library; and / or clean the special characters contained in the text dictionary through the HTML parsing library. The key information verification tool is mainly used to verify the extraction results of the key information extraction tool. Specifically, it verifies the accuracy of the content of the extraction result dictionary by calling a verification model; the verification model returns the correctly extracted content in the extraction result dictionary in the original format, removes the incorrectly extracted information in the extraction result dictionary, and adds the incorrectly extracted content as negative samples to the training data set; among them, the verification model is constructed based on the Bert pre-trained model and fine-tuned through doccano-annotated classification data. Corresponding to the extraction tool, the verification tool can also be subdivided according to specific extraction targets, such as a technical problem verification tool, a technical means verification tool, a technical effect verification tool, etc.

[0025] The model optimization tool mainly uses the negative samples verified by the verification tool to update and optimize the extraction tool and the verification tool in real time. When the number of samples in the training data set changes, all the samples in the training data set are divided into a training set and a test set according to the language type and proportion, such as 8:2; the data path for saving the training set, the training path for saving the test set, and the path of the model to be trained are used as the parameter inputs of the script function of the model optimization program. The model to be trained includes an extraction model and a verification model; start the model optimization program until the model optimization is completed to obtain the optimized model. The data storage tool is used to connect to the data writing interface of the database and store the content of the extraction result dictionary in the database. When actually saving, the identifier of the technical document can be used as the key value, and the content of the corresponding extraction result dictionary can be used as the corresponding value for saving, or the task label of the extraction target can be used as the key value, and the content of the corresponding extraction result dictionary can be used as the corresponding value for saving.

[0026] S20. The agent invokes the large language model to decompose the extraction actions corresponding to each extraction target according to each extraction target, forming at least one extraction subtask of the extraction target.

[0027] After the agent obtains the text information input by the user, it invokes the large language model to parse the natural language input of the user, determines all extraction targets in the user's text information and the extraction actions of each extraction target, and decomposes each extraction action to form at least one extraction subtask of the extraction target, ensuring that after all extraction subtasks are executed, the operations expected by the user for the extraction target can be completed.

[0028] For example, for the following text information input by the user: "Extract the technical problems and technical means of the patent with the publication number CN12345678A, and save the extracted keywords of the technical problems to the database", the large language model can parse that the extraction targets this time are technical problems and technical means. The operations to be performed on the technical problems include obtaining the technical text containing the technical problems, extracting the key information of the technical problems, and saving the key information of the technical problems, while the operations to be performed on the technical means include obtaining the technical text containing the technical means and extracting the key information of the technical means.

[0029] S30. The large language model determines at least one tool to be invoked for each extraction subtask according to the tool description, and forms the invocation order between all tools to be invoked according to the logical relationship between the extraction subtasks, so as to form a task scheduling sequence.

[0030] After the large language model determines all extraction subtasks, it will match the extraction subtasks through the tool description according to all the tools it has, and determine at least one tool to be invoked for each extraction subtask to complete the subtask. Further, combining the logical relationship between the extraction subtasks, sort all the tools to be invoked to form a task scheduling sequence, which is used as the task execution order for actually executing and solving the user's requirements.

[0031] In some embodiments, when a tool is built, each tool has its corresponding natural language task description, which can be used as the basis for calling the large language model. Moreover, when describing its own functions, the tool description of the current tool can also describe its association with other tools, so as to serve as the basis for the large language model to determine the tool to be called and the calling order between subsequent tools to be called. For example, the tool description of the key information extraction tool can be "used to extract key information from a document", and the tool description of the key information verification tool can be "used to verify whether the result extracted by the key information extraction tool is accurate". At this time, although the user does not emphasize the need to verify the extracted content in the text information, when actually calling, the large language model will combine the tool description of the verification tool and add the verification tool after the extraction tool to achieve better extraction operations.

[0032] In the process of the large language model actually determining the tool to be called and the calling order, even for the purpose of completing the same extraction subtask, there may be multiple different selection results of the tool to be called and multiple different calling orders. The large language model will combine the ReAct mechanism to select the tool to be called and its calling order that it deems the most reasonable, forming a task orchestration sequence.

[0033] S40, the large language model sequentially calls the tools to be called in the task orchestration sequence to process the technical document information until the extraction actions for all extraction targets are completed, forming an output result and presenting it to the user.

[0034] After the task orchestration sequence is formed, the large language model will sequentially call the tools to be called according to the task orchestration sequence to process the technical document information, so as to extract the key information that the user hopes to obtain. When all the tools to be called in the task orchestration sequence have been called, the large language model determines that all the extraction actions for all extraction targets have been completed, and presenting the obtained output result to the user can complete the requirements of the user in this conversation. It should be noted that the output result in this embodiment usually refers to the extraction result of key information, or the processing and presentation result of specific content involved in the user's text information, rather than the execution result of each tool.

[0035] In the actual implementation process, the large language model combines the ReAct mechanism to observe and think about the overall execution process of the task orchestration sequence. For example, when calling the current tool, it observes the execution result of the current tool to determine whether the execution result of the current tool meets the input requirements of the next tool in the task orchestration sequence that has a sequential execution logic with the current tool. If the execution result of the current tool meets the input requirements of the next tool, the large language model uses the execution result of the current tool as the input data for the next tool; if the execution result of the current tool does not meet the input requirements of the next tool, the large language model re-determines the next tool based on the execution result and the tool description, or the large language model re-forms the task orchestration sequence based on the execution result; the large language model makes tool calls according to the re-determined next tool or the re-determined task orchestration sequence until the extraction actions for all extraction targets are completed, forming an output result and presenting it to the user.

[0036] For example, when a user expects to obtain the technical problems and technical means in a patent document, the extraction targets confirmed by the large language model include technical problems and technical means, corresponding to the extraction actions for technical problems and the extraction actions for technical means, and the following task orchestration sequence is formed: data acquisition tool, data preprocessing tool, technical problem key information extraction tool, technical problem key information verification tool, technical means key information extraction tool, and technical means key information verification tool. Among them, the description of the data preprocessing tool may emphasize that its role is to preprocess the acquisition result of the data acquisition tool, so there is a sequential execution logic between the data acquisition tool and the data preprocessing tool. Similarly, the technical problem key information verification tool also needs to be called after the technical problem key information extraction tool is called, so there is also a sequential execution logic between the technical problem key information extraction tool and the technical problem key information verification tool. There is also a sequential execution logic between the technical means key information extraction tool and the technical means key information verification tool. Therefore, after the large language model executes the tool with a higher execution order, it will observe its execution result and make subsequent calls when the input requirements of the tool with a lower execution order are met. However, there is no such sequential execution logic between the technical problem key information verification tool and the technical means key information extraction tool. Therefore, even though they are also executed in a specific order, the large language model will not observe whether the execution result of the technical problem key information verification tool meets the input requirements of the technical means key information extraction tool.

[0037] In some embodiments, the agent takes a complete piece of text information input by the user in the operable interface as one conversation and performs corresponding executions. After presenting the output result to the user, the text information, the task orchestration sequence, and the output result in this conversation are stored in the short-term memory as historical records, so that the agent can remember the previous communication content when performing subsequent conversation tasks and provide continuous and personalized responses.

[0038] In this embodiment, by constructing an agent and using a large language model as the brain of the agent, the extraction operation information input by the user is understood and the task is disassembled. Subsequently, the agent thinks independently and calls the corresponding tools to complete the key information extraction of the technical document information. Without being executed according to a fixed process setting, the agent can dynamically generate the task process that best meets the user's needs and output the result. At the same time, it can meet the information extraction of any type of technical document, greatly improving flexibility and scalability.

[0039] Based on the same inventive concept, the second embodiment of the present disclosure provides an apparatus for extracting key information from technical documents based on an agent. The schematic structural diagram is as Figure 3 shown, and mainly includes: an interaction module 10 and an operation module 20; wherein, the interaction module 10 is used to call the agent to receive the text information input by the user. The text information at least includes: technical document information and extraction operation information. The extraction operation information at least includes: at least one extraction target and the extraction action performed on the extraction target; the operation module 20 is used to call the large language model to disassemble the extraction action corresponding to each extraction target according to each extraction target to form at least one extraction subtask of the extraction target; the large language model determines at least one tool to be called for each extraction subtask according to the tool description; the large language model forms the call order between all the tools to be called according to the logical relationship between the extraction subtasks to form a task orchestration sequence; the large language model sequentially calls the tools to be called in the task orchestration sequence to process the technical document information until the extraction actions for all extraction targets are completed, forms an output result and presents it to the user.

[0040] In some embodiments, when the large language model invokes the current tool, it observes the execution result of the current tool to determine whether the execution result of the current tool meets the input requirements of the next tool in the task orchestration sequence that has a sequential execution logic with the current tool; in the case where the execution result of the current tool meets the input requirements of the next tool, the large language model uses the execution result of the current tool as the input data for the next tool; in the case where the execution result of the current tool does not meet the input requirements of the next tool, the large language model re-determines the next tool based on the execution result and the tool description, or the large language model reformulates the task orchestration sequence based on the execution result; the large language model makes tool invocations according to the re-determined next tool or the re-determined task orchestration sequence until the extraction actions for all extraction targets are completed, forming an output result and presenting it to the user.

[0041] In some embodiments, the device further includes an agent construction module 30, which is mainly used for: constructing a tool library, where the tool library at least includes: a data acquisition tool, a key information extraction tool; constructing a large language model and adding the tool library to the skill list of the large language model; using the large language model and the tool library as parameters of the LangChain framework to construct an agent.

[0042] Specifically, the data acquisition tool is used for: obtaining a technical text according to the technical document information, parsing and extracting the text content corresponding to different chapters of the technical text, and splicing and assembling the text content of different chapters into at least one text dictionary according to the extraction target according to a preset rule; where, in the case where the technical document information includes the technical text, the data acquisition tool directly obtains the technical text, and in the case where the technical document information is an identifier used to represent the technical text, the data acquisition tool calls a data acquisition interface according to the identifier to obtain the technical text; the key information extraction tool is used for: forming a task label for each extraction target; performing key information extraction on the text dictionary matching the extraction target by calling an extraction model, and encapsulating the extraction result and the task label corresponding to the extraction target into an extraction result dictionary. The extraction model uses a general information extraction model as the base model and is fine-tuned by doccano-annotated multilingual classification data.

[0043] In some embodiments, the tool library further includes at least one of a data preprocessing tool, a key information verification tool, a data storage tool, and a model optimization tool; wherein, the data preprocessing tool is used to: clean the HTML tags included in the text dictionary by a mixed cleaning method of regular expressions and an HTML parsing library; and / or, clean the special characters included in the text dictionary by the HTML parsing library; the key information verification tool is used to: verify the accuracy of the content of the extraction result dictionary by calling a verification model; the verification model returns the correctly extracted content in the extraction result dictionary in the original format, removes the incorrectly extracted information in the extraction result dictionary, and adds the incorrectly extracted content as negative samples to the training data set; wherein, the verification model is constructed based on the Bert pre-trained model and fine-tuned by doccano-annotated classification data; the model optimization tool is used to: when the number of samples in the training data set changes, divide all the samples in the training data set into a training set and a test set according to the language type and proportion; input the data path for saving the training set, the training path for saving the test set, and the model path to be trained as the parameters of the model training into the script function of the model optimization program, and the model to be trained includes an extraction model and a verification model; start the model optimization program until the model optimization ends to obtain an optimized model; the data storage tool is used to: interface with the data writing interface of the database and store the content of the extraction result dictionary in the database.

[0044] In some embodiments, the operation module 20 is further configured to store the text information, task choreography sequence, and output result in the current conversation as historical records in the short-term memory.

[0045] In this embodiment, by constructing an agent and using a large language model as the brain of the agent, the extraction operation information input by the user is understood and the task is disassembled. Subsequently, the agent thinks independently and calls the corresponding tools to complete the key information extraction of the technical document information. There is no need to execute according to a fixed process setting. The agent can dynamically generate the task process that best meets the user's needs and output the result. At the same time, it can meet the information extraction of any type of technical document, greatly improving the flexibility and scalability.

[0046] Based on the same inventive concept, the third embodiment of the present disclosure provides a storage medium storing a computer program, characterized in that when the computer program is executed by a processor, it implements the steps of the method for extracting key information of technical documents based on an agent according to the first embodiment of the present disclosure.

[0047] Based on the same inventive concept, the fourth embodiment of the present disclosure provides an electronic device including at least a memory and a processor, with a computer program stored on the memory, characterized in that when the processor executes the computer program on the memory, it implements the steps of the method for extracting key information of technical documents based on an agent according to the first embodiment of the present disclosure.

[0048] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them; although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A method for extracting key information from technical documents based on an agent, characterized in that: include: The agent receives text information input by a user, wherein the text information includes at least technical document information and extraction operation information, wherein the extraction operation information includes at least at least one extraction target and an extraction action performed on the extraction target; The agent calls the large language model to decompose the extraction action corresponding to each extraction target according to the extraction target to form at least one extraction subtask of the extraction target; The large language model determines at least one tool to be called for each of the extracted subtasks according to the tool description, and forms a calling sequence among all the tools to be called according to the logical relationship between the extracted subtasks to form a task arrangement sequence; The large language model sequentially calls the tools to be called in the task arrangement sequence to process the technical document information until the extraction actions for all the extraction targets are completed, and output results are formed and displayed to the user.

2. The method for extracting key information from technical documents according to claim 1, characterized in that: The large language model sequentially calls the tools to be called in the task arrangement sequence to process the technical document information until the extraction actions of all the extraction targets are completed, and output results are formed and displayed to the user, including: When calling the current tool, the large language model observes the execution result of the current tool to determine whether the execution result of the current tool meets the input requirement of the next tool in the task scheduling sequence that has a sequential execution logic with the current tool; When the execution result of the current tool meets the input requirement of the next tool, the large language model uses the execution result of the current tool as input data of the next tool; In the case that the execution result of the current tool does not meet the input requirement of the next tool, the large language model redetermines the next tool according to the execution result and the tool description, or the large language model re-forms the task arrangement sequence according to the execution result; The large language model calls the tool according to the re-determined next tool or the re-determined task arrangement sequence until the extraction actions for all the extraction targets are completed, and the output results are formed and displayed to the user.

3. The method for extracting key information from technical documents according to claim 1, characterized in that: The agent is constructed based on the following steps: Constructing a tool library, wherein the tool library at least includes: a data acquisition tool and a key information extraction tool; Building a large language model, and adding the tool library to a skill list of the large language model; The large language model and the tool library are used as parameters of the LangChain framework to build an intelligent agent.

4. The method for extracting key information from technical documents according to claim 3, characterized in that: The data acquisition tool is used to: acquire technical text according to the technical document information, parse and extract text contents corresponding to different chapters of the technical text, and splice and assemble the text contents of different chapters into at least one text dictionary according to the extraction target and preset rules; wherein, in the case where the technical document information includes the technical text, the data acquisition tool directly acquires the technical text, and in the case where the technical document information is an identifier for characterizing the technical text, the data acquisition tool calls a data acquisition interface according to the identifier to acquire the technical text; The key information extraction tool is used to: form a task label for each extraction target; extract key information from a text dictionary matching the extraction target by calling an extraction model, and encapsulate the extraction result and the task label corresponding to the extraction target into an extraction result dictionary.

5. The method for extracting key information from technical documents according to claim 4, characterized in that: The extraction model uses the general information extraction model as the base model and is fine-tuned by annotating multilingual classification data with doccano.

6. The method for extracting key information from technical documents according to claim 4, characterized in that: The tool library also includes at least one of a data preprocessing tool, a key information verification tool, a data warehousing tool, and a model optimization tool; wherein, The data preprocessing tool is used to: clean the HTML tags contained in the text dictionary by a mixed cleaning method of regular expressions and HTML parsing library; and / or clean the special characters contained in the text dictionary by the HTML parsing library; The key information verification tool is used to: verify the accuracy of the content of the extraction result dictionary by calling the verification model; the verification model returns the correctly extracted content in the extraction result dictionary in the original format, removes the erroneously extracted information in the extraction result dictionary, and adds the erroneously extracted content as a negative sample to the training data set; wherein, the verification model is built based on the Bert pre-training model and fine-tuned through doccano annotated classification data; The model optimization tool is used to: when the number of samples in the training data set changes, divide all samples in the training data set into a training set and a test set according to language type and proportion; input the data path for storing the training set, the training path for storing the test set, and the model path to be trained as parameters for model training into the script function of the model optimization program, wherein the model to be trained includes an extraction model and a verification model; start the model optimization program until the model optimization is completed, and obtain the optimized model; The data warehousing tool is used to connect to the data writing interface of the database and store the contents of the extraction result dictionary in the database.

7. The method for extracting key information from technical documents according to any one of claims 1 to 6, characterized in that: Also includes: The agent stores the text information, the task scheduling sequence and the output results in this conversation as historical records in short-term memory.

8. An agent-based device for extracting key information from technical documents, characterized in that: include: An interactive module, used to call an agent to receive text information input by a user, wherein the text information includes at least technical document information and extraction operation information, and the extraction operation information includes at least at least one extraction target and an extraction action performed on the extraction target; An operation module is used to call the large language model to decompose the extraction action corresponding to each extraction target according to the extraction target to form at least one extraction subtask of the extraction target; the large language model determines at least one tool to be called for each extraction subtask according to the tool description; the large language model forms a calling order between all the tools to be called according to the logical relationship between the extraction subtasks to form a task scheduling sequence; the large language model sequentially calls the tools to be called in the task scheduling sequence to process the technical document information until the extraction actions of all the extraction targets are completed, and output results are formed and displayed to the user.

9. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the agent-based technical document key information extraction method described in any one of claims 1 to 7 are implemented.

10. An electronic device, comprising at least a memory and a processor, wherein a computer program is stored in the memory, wherein: The processor implements the steps of the agent-based technical document key information extraction method described in any one of claims 1 to 7 when executing the computer program on the memory.

Images