Data management method and system based on large model agent

Abstract

The invention discloses a data governance method and system based on a large model agent, and the method is executed by a model context protocol client, and comprises the steps: sending a call request to a preset first model context protocol server, and obtaining to-be-governed unstructured text data; calling a first processing module, and utilizing a large language model to generate a metadata definition scheme containing field names and types on the basis of semantic contents of the unstructured text data; calling a second processing module, taking the unstructured text data and the metadata definition scheme as a constraint context, mapping and extracting key element values from the text data based on the scheme, and generating a structured data result; and sending a calling request to a preset second model context protocol server, and writing the structured data result into the target database. According to the method, the mode of depending on manual definition of metadata is changed through an intelligent means, the unstructured data processing capacity is remarkably improved, and the treatment complexity and the operation and maintenance burden are greatly reduced.

Description

Technical Field

[0001] This invention relates to the field of data governance technology, and mainly to a data governance method and system based on large model intelligent agents. Background Technology

[0002] With the deepening of digital transformation, massive amounts of unstructured data (such as announcements, legal documents, medical records, and evidence) have been generated in fields such as finance, healthcare, government and enterprise, and industry. Currently, the governance of this type of data faces serious challenges.

[0003] Metadata definition relies on manual processes: Traditional governance processes require defining metadata standards first, followed by data cleaning. When faced with unstructured documents with varying formats, manually sorting fields and defining data dictionaries is inefficient and prone to omissions or inaccuracies in identifying key elements.

[0004] Rule engines have significant limitations: existing governance tools mostly rely on predefined regular expressions, UDF functions, or hard-coded rules. For unstructured data with semantic ambiguity or inconsistent formats, rigid code rules are difficult to cover all scenarios, and the maintenance cost of rules is extremely high once business changes.

[0005] Insufficient intelligence: Traditional ETL tools lack semantic understanding capabilities and cannot adaptively adjust extraction strategies based on document content, making it difficult to achieve dynamic and automated data governance.

[0006] Therefore, there is an urgent need for an adaptive data governance solution that can integrate large model technology, reshape the data governance process, and achieve automatic definition of metadata and intelligent extraction of key elements. Summary of the Invention

[0007] The purpose of this invention is to provide a data governance method and system based on large model intelligent agents, which aims to solve the problems of high dependence on manual intervention, difficulty in rule maintenance, and weak unstructured data processing capabilities in traditional data governance.

[0008] To achieve the above objectives, this invention provides a data governance method based on large model intelligent agents. The method is executed by a model context protocol client and includes:

[0009] Send a call request to the preset first model context protocol server to obtain the unstructured text data to be processed; wherein, the first model context protocol server connects to the local or remote file system;

[0010] The first processing module is invoked, and unstructured text data is input into the first processing module. The first processing module invokes the large language model and generates a metadata definition scheme containing field names, field types, and field descriptions based on the semantic content of the unstructured text data.

[0011] The second processing module is invoked, and the unstructured text data and metadata definition scheme are used as constraint contexts and input into the second processing module. Based on the metadata definition scheme, the second processing module maps and extracts the corresponding key element values ​​from the unstructured text data to generate structured data results.

[0012] A call request is sent to the preset second model context protocol server to write the structured data results into the target database; wherein, the second model context protocol server connects to the target database.

[0013] Through the above technical solutions, the inefficient model of relying on manually defined metadata is completely changed by intelligent means, significantly improving the processing capability of unstructured document data; at the same time, a more intelligent and adaptive approach is adopted to greatly reduce the complexity of governance and the burden of operation and maintenance.

[0014] Furthermore, a call request is sent to the first model context protocol server to obtain the unstructured text data to be processed. Specifically, this includes: establishing a connection with the first model context protocol server and sending a file parsing request; the first model context protocol server connects to the local or remote file system, reads and parses the unstructured text data, and returns the result.

[0015] Through the above technical solutions, based on the Model Context Protocol (MCP) tool plugin encapsulation, file reading and parsing and result data storage are realized, enhancing application scalability.

[0016] Furthermore, the first processing module is configured as a metadata analysis and definition agent; wherein, the construction process of the metadata analysis and definition agent includes:

[0017] Configure the first role definition prompt word. The first role definition prompt word is a user-defined prompt word for the scenario defined by metadata analysis.

[0018] Construct the first task instruction to constrain the large language model to recognize entities, elements and relationships in the text; set the first output constraint to constrain the large language model to output metadata definitions in JSON format, and the metadata definitions shall include at least field names, field types, field lengths and field descriptions;

[0019] The first role definition prompt and the first task instruction are combined to form the first system prompt.

[0020] By using the above technical solutions, key information in files can be extracted and corresponding metadata information can be defined by using prompt word engineering to guide large model analysis. This solves the problems of low efficiency and omission or inaccuracy in the original data governance, which required manual definition of metadata.

[0021] Furthermore, the second processing module is configured as a key element extraction agent; wherein, the construction process of the key element extraction agent includes:

[0022] Configure a second role definition prompt word. The second role definition prompt word is a role prompt word for the user-defined metadata extraction information scenario.

[0023] Construct the second task instruction, receive the metadata definition scheme as input constraints, and constrain the large language model to traverse unstructured text data, match and extract specific numerical values ​​or text fragments corresponding to the metadata definition scheme, and output them in the form of key-value pairs;

[0024] The second system prompt is created by combining the second role definition prompt and the second task instruction.

[0025] By using the above technical solutions, data corresponding to key elements can be extracted based on metadata information, and data governance-related work can be completed, effectively avoiding misjudgment or omission of key elements.

[0026] Furthermore, before inputting the unstructured text data into the first processing module, the process includes: calling an intent recognition agent to classify the unstructured text data and determine the business domain type of the data; and matching the corresponding industry standard knowledge base as auxiliary input to the first processing module based on the business domain type.

[0027] The above technical solutions leverage the large model's ability to understand the semantics of unstructured data and its powerful knowledge base to expand data dimensions; at the same time, they ensure that the processed file structure and content are relatively uniform, avoiding excessively broad metadata information due to large spans.

[0028] Furthermore, a call request is sent to the second model context protocol server to write the structured data results into the target database. Specifically, this includes: establishing a connection with the second model context protocol server and sending a data write request; the second model context protocol server connecting to the target database and performing the structured data storage operation.

[0029] Furthermore, the methods also include:

[0030] After dynamically generating the metadata definition scheme, the format validity of the metadata definition scheme output by the first processing module is determined. If the determination is invalid, the exception handling module is called to assemble the exception information and terminate the current governance process.

[0031] After generating structured data results, the structured data results output by the second processing module are successfully extracted. If the extracted content does not conform to the metadata definition scheme, the exception handling module is called to assemble exception information and terminate the current governance process.

[0032] After storing the data in the target database, the system receives the write status returned by the database. If the write operation fails, the system calls the exception handling module to assemble the write exception information.

[0033] The above technical solutions verified whether the metadata-related definitions output by the metadata analysis were reasonable and comprehensive, and verified whether the key elements were extracted according to the defined metadata, thus ensuring the accuracy and stability of the data governance process.

[0034] Secondly, this invention provides a data governance system based on a large-scale intelligent agent, the system comprising:

[0035] The first model context protocol server encapsulates file parsing tools and is configured to connect to local or remote file systems.

[0036] The second model context protocol server encapsulates a database connection tool and is configured to connect to the target database.

[0037] A Model Context Protocol (MGP) client is configured to communicate with a first MGP server and a second MGP server.

[0038] The Model Context Protocol (MCP) clients include:

[0039] The first processing module is configured to use a large language model to generate a metadata definition scheme that includes field names, field types and field descriptions based on the semantic content of unstructured text data.

[0040] The second processing module is configured to receive unstructured text data and metadata definition schemes as constraint contexts; based on the metadata definition schemes, it maps and extracts the corresponding key element values ​​from the unstructured text data to generate structured data results.

[0041] Thirdly, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the above-described method.

[0042] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0043] Compared with the prior art, the beneficial effects of the present invention are:

[0044] By employing intelligent methods, the system completely transforms the inefficient model that relies on manually defined metadata, significantly improving the processing capabilities of unstructured document data and avoiding misjudgment or omission of key elements. At the same time, it breaks through the limitations of predefined rule engines, endowing the system with powerful semantic understanding capabilities to accurately process unstructured or ambiguous data. Furthermore, it abandons traditional code, regular expressions, UDF functions, and other high-maintenance-cost tools, adopting a more intelligent and adaptive approach to greatly reduce governance complexity and operational burden. Attached Figure Description

[0045] The accompanying drawings are included to provide a further understanding of the embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and, together with the description, serve to explain the principles of the invention. Other embodiments and many anticipated advantages of the embodiments will be readily recognized as they become better understood through reference to the following detailed description. Elements in the drawings are not necessarily to scale. The same reference numerals refer to corresponding similar parts.

[0046] Figure 1 This is a flowchart of a data governance method based on a large model intelligent agent according to an embodiment of the present invention;

[0047] Figure 2 This is a framework diagram of a data governance system based on a large model intelligent agent according to an embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of the structure of a computer device suitable for implementing electronic devices according to embodiments of the present invention. Detailed Implementation

[0049] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0050] Firstly, reference Figure 1 This embodiment provides a data governance method based on large model intelligent agents. Figure 1 As shown, this method achieves closed-loop governance of unstructured data through agent collaboration and model context protocol (MCP).

[0051] It should be noted that the Model Context Protocol (MCP) is an open standard protocol for standardizing the interaction between large language models (LLMs) and external data sources and tools.

[0052] The method in this embodiment is executed by the Model Context Protocol client and specifically includes the following steps:

[0053] Step S100: File parsing;

[0054] Specifically, file parsing includes: sending a call request to a preset first model context protocol server to obtain the unstructured text data to be processed; wherein, the first model context protocol server connects to a local or remote file system.

[0055] Preferably, a call request is sent to the first model context protocol server to obtain the unstructured text data to be processed. Specifically, this includes: establishing a connection with the first model context protocol server and sending a file parsing request; the first model context protocol server connecting to a local or remote file system, reading and parsing the unstructured text data, and returning the results. Based on the MCP tool plugin encapsulation, file reading, parsing, and result data storage are achieved, enhancing application scalability.

[0056] Optionally, before inputting unstructured text data into the first processing module, an initialization preparation phase is also included: understanding relevant industry standards and fully recognizing key elements within those standards. This involves a preliminary understanding of the unstructured text data file, calling an intent recognition agent to classify the unstructured text data, and determining the data's business domain type, such as: announcements, investigations, or evidence.

[0057] Specifically, based on the type of business domain, a corresponding industry-standard knowledge base is matched as auxiliary input for the first processing module. The large model's ability to understand the semantics of unstructured data and its powerful knowledge base capabilities are leveraged to expand the data dimensions; at the same time, the structure and content of the processed files are kept relatively uniform to avoid excessively broad metadata information due to large spans.

[0058] Step S200: Metadata Analysis and Definition (First Processing Module);

[0059] Specifically, metadata analysis and definition include: calling the first processing module, inputting unstructured text data into the first processing module, and the first processing module calling the large language model to generate a metadata definition scheme containing field names, field types and field descriptions based on the semantic content of the unstructured text data.

[0060] Specifically, the first processing module is configured as a metadata analysis definition agent.

[0061] The metadata analysis definition of the intelligent agent construction process includes:

[0062] Configure the first role definition prompt word. The first role definition prompt word is a user-defined prompt word for the scenario defined by metadata analysis.

[0063] Construct the first task instruction to constrain the large language model to recognize entities, elements and relationships in the text; set the first output constraint to constrain the large language model to output metadata definitions in JSON format, and the metadata definitions shall include at least field names, field types, field lengths and field descriptions;

[0064] The first role definition prompt and the first task instruction are concatenated to form the first system prompt. This process uses prompt engineering to guide large-scale model analysis to extract key information from the files and define corresponding metadata information, solving the problems of low efficiency and omissions or inaccuracies in the original data governance, which required manual definition of metadata.

[0065] As an example, the specific content of the first system prompt is as follows:

[0066] "###Role

[0067] You are a senior expert with dual professional backgrounds:

[0068] **Text Analysis Expert:** Skilled at identifying key entities, elements, relationships, and other information from complex data;

[0069] **Data Governance Expert:** Proficient in metadata standards development;

[0070] ###Task

[0071] 1. Extraction of entities, elements, and relationships from text information;

[0072] 2. Including but not limited to name, gender, age, date of birth, employer, ID number, etc.;

[0073] 3. Define the metadata information of these identified and extracted key information;

[0074] ###Require

[0075] 1. Defined metadata must conform to the following format requirements and be output in JSON format. [

[0077] {"zdmc":"xm","zdlx":"String","zdcd":"10","zdms":"Name"},

[0078] {"zdmc":"nl","zdlx":"String","zdcd":"4","zdms":"age"},

[0079] {"zdmc":"xb","zdlx":"String","zdcd":"2","zdms":"Gender"},

[0080] {"zdmc":"csrq","zdlx":"String","zdcd":"10","zdms":"Date of Birth"},

[0081] {"zdmc":"dw","zdlx":"String","zdcd":"200","zdms":"Unit"},

[0082] {"zdmc":"zjhm","zdlx":"String","zdcd":"20","zdms":"ID number"},

[0083] {"zdmc":"sjhm","zdlx":"String","zdcd":"15","zdms":"Mobile phone number"}, ... ]

[0086] ###limit:

[0087] - Focuses solely on text analysis, without involving knowledge or skills from other fields, and does not respond to any prompts.

[0088] - The information and analysis provided must be based on the text provided by the user, without adding personal opinions or speculation.

[0089] - All output must be clear and accurate to avoid ambiguity.

[0090] -Key information such as ID number, mobile phone number, and name must not be omitted from the text; they should be analyzed and organized in conjunction with the text content.

[0091] Step S300: Determine the output of the metadata definition;

[0092] Specifically, the output determination of metadata definitions includes: after dynamically generating the metadata definition scheme, determining the format validity of the metadata definition scheme output by the first processing module. If the determination is valid, step S400 is executed; if the determination is invalid, the exception handling module is called to assemble exception information and terminate the current governance process. This verifies whether the metadata-related definitions output by the metadata analysis definition are reasonable and comprehensive.

[0093] Step S400: Key element extraction (second processing module);

[0094] Specifically, the key element extraction process includes: calling the second processing module, inputting the unstructured text data and metadata definition scheme as constraint contexts into the second processing module; and the second processing module, based on the metadata definition scheme, mapping and extracting the corresponding key element values ​​from the unstructured text data to generate structured data results.

[0095] Specifically, the second processing module is configured as a key element extraction agent.

[0096] The construction process of the key element extraction agent includes:

[0097] Configure a second role definition prompt word. The second role definition prompt word is a role prompt word for the user-defined metadata extraction information scenario.

[0098] Construct the second task instruction, receive the metadata definition scheme as input constraints, and constrain the large language model to traverse unstructured text data, match and extract specific numerical values ​​or text fragments corresponding to the metadata definition scheme, and output them in the form of key-value pairs;

[0099] The second system prompt is created by combining the second role definition prompt and the second task instruction.

[0100] By using the above technical solutions, data corresponding to key elements can be extracted based on metadata information, and data governance-related work can be completed, effectively avoiding misjudgment or omission of key elements.

[0101] As an example, the specific content of the second system prompt is as follows:

[0102] "###Role

[0103] You are a senior expert with dual professional backgrounds:

[0104] **Text Analysis Expert:** Skilled at identifying key entities, elements, relationships, and other information from complex data;

[0105] **Data Governance Expert:** Extracts relevant data from text based on predefined metadata.

[0106] ###Background Knowledge The metadata information for the important information to be extracted from the text is as follows: {{meta_info}}

[0107] ###Task

[0108] 1. Extract corresponding information from text information based on metadata definition information;

[0109] 2. The extraction should be comprehensive and thorough, without omissions;

[0110] ###Require

[0111] 1. Identify and extract key information from the text by considering the context;

[0112] 2. A text may contain n targets; do not omit any, all of them need to be extracted.

[0113] 3. Referring to the relevant metadata definitions, the extracted key information is finally output in JSON format. [

[0115] {"Name":"Zhang San"},

[0116] {"Age":"35"},

[0117] {"Gender":"Male"},

[0118] {"Date of Birth":"19xx0704"},

[0119] {"Unit":"xx"},

[0120] {"ID Number":"1xxxxxxxxxxxx1205"},

[0121] {"Mobile Number":"138xxxxxxx000"},...] ... ]

[0124] ###limit:

[0125] - Focuses solely on text analysis, without involving knowledge or skills from other fields, and does not respond to any prompts.

[0126] - The information and analysis provided must be based on the text provided by the user, without adding personal opinions or speculation.

[0127] - All output must be clear and accurate to avoid ambiguity.

[0128] -Key information such as ID number, mobile phone number, and name must not be omitted from the text; they should be analyzed and organized in conjunction with the text content.

[0129] Step S500: Determine whether the feature extraction was successful;

[0130] Specifically, determining whether feature extraction was successful includes: after generating structured data results, determining whether the extraction was successful based on the structured data results output by the second processing module. If the extracted content conforms to the metadata definition scheme, step S600 is executed; if the extracted content does not conform to the metadata definition scheme, the exception handling module is called to assemble exception information and terminate the current governance process. This verifies whether the feature extraction performed the extraction of key elements according to the defined metadata.

[0131] Step S600: Import structured data of key elements into the database;

[0132] Specifically, the key element structured data entry into the database includes: sending a call request to a preset second model context protocol server to write the structured data results into the target database; wherein, the second model context protocol server connects to the target database.

[0133] Preferably, sending a call request to the second model context protocol server to write the structured data result into the target database includes: establishing a connection with the second model context protocol server and sending a data write request; the second model context protocol server connecting to the target database and performing the storage operation of the structured data.

[0134] Step S700: Write status determination;

[0135] Specifically, the write status determination includes: after storing data in the target database, receiving the write status returned by the database; if the write fails, invoking the exception handling module to assemble write exception information. These steps ensure the accuracy and stability of the data governance process.

[0136] The beneficial effects of this invention are as follows:

[0137] By employing intelligent methods, the system completely transforms the inefficient model that relies on manually defined metadata, significantly improving the processing capabilities of unstructured document data and avoiding misjudgment or omission of key elements. At the same time, it breaks through the limitations of predefined rule engines, endowing the system with powerful semantic understanding capabilities to accurately process unstructured or ambiguous data. Furthermore, it abandons traditional code, regular expressions, UDF functions, and other high-maintenance-cost tools, adopting a more intelligent and adaptive approach to greatly reduce governance complexity and operational burden.

[0138] Secondly, this embodiment provides a data governance system based on a large-scale intelligent agent model. For example... Figure 2 As shown, the system includes:

[0139] First Model Context Protocol Server 100 (File Reading mcp-server): Encapsulates file parsing tools and is configured to connect to local or remote file systems 400.

[0140] The second model context protocol server 200 (database operation mcp-server): encapsulates database connection tools and is configured to connect to the target database 500.

[0141] Specifically, the target database 500 includes MongoDB and other databases.

[0142] Model Context Protocol Client 300 (Large Model mcp-client): Configured for communication connections with the first Model Context Protocol Server 100 and the second Model Context Protocol Server 200;

[0143] The Model Context Protocol client 300 includes:

[0144] The first processing module (metadata analysis and definition agent) is configured to use a large language model to generate a metadata definition scheme that includes field names, field types and field descriptions based on the semantic content of unstructured text data.

[0145] The second processing module (Key Element Information Extraction Agent) is configured to receive unstructured text data and metadata definition schemes as constraint contexts; based on the metadata definition schemes, it maps and extracts the corresponding key element values ​​from the unstructured text data to generate structured data results.

[0146] The following is for reference. Figure 3 It shows a schematic diagram of the structure of a computer system suitable for implementing an electronic device according to embodiments of the present invention. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0147] like Figure 3 As shown, the computer system includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 302 or programs loaded from storage section 308 into random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the system 300. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0148] The following components are connected to I / O interface 305: an input section 306 including a keyboard, mouse, etc.; an output section 307 including a liquid crystal display (LCD) and speakers, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN card and a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to I / O interface 305 as needed. A removable medium 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 310 as needed so that computer programs read from it can be installed into storage section 308 as needed.

[0149] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by central processing unit (CPU) 301, it performs the functions defined in the methods of the present invention. It should be noted that the computer-readable storage medium of the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable storage medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0150] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0151] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0152] The modules described in the embodiments of the present invention can be implemented in software or in hardware.

[0153] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the following: sending a call request to a preset first model context protocol server to obtain unstructured text data to be processed; wherein the first model context protocol server is connected to a local or remote file system; calling a first processing module to input the unstructured text data into the first processing module, the first processing module calling a large language model to generate a metadata definition scheme containing field names, field types, and field descriptions based on the semantic content of the unstructured text data; calling a second processing module to input the unstructured text data and the metadata definition scheme as a constraint context into the second processing module; the second processing module mapping and extracting corresponding key element values ​​from the unstructured text data based on the metadata definition scheme to generate structured data results; and sending a call request to a preset second model context protocol server to write the structured data results into a target database; wherein the second model context protocol server is connected to the target database.

[0154] The specific embodiments of the present invention have been described above, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0155] In the description of this invention, it should be understood that the terms "upper," "lower," "inner," "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The simple fact that certain measures are recited in mutually different dependent claims does not indicate that combinations of these measures cannot be used for improvement. Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. A data governance method based on large-model intelligent agents, characterized in that, The method is executed by the Model Context Protocol client and includes: Send a call request to a preset first model context protocol server to obtain the unstructured text data to be processed; wherein, the first model context protocol server is connected to a local or remote file system; The first processing module is invoked, and the unstructured text data is input into the first processing module. The first processing module invokes the large language model and generates a metadata definition scheme containing field names, field types and field descriptions based on the semantic content of the unstructured text data. The second processing module is invoked, and the unstructured text data and the metadata definition scheme are used as constraint contexts and input into the second processing module. Based on the metadata definition scheme, the second processing module maps and extracts the corresponding key element values ​​from the unstructured text data to generate structured data results. A call request is sent to a preset second model context protocol server to write the structured data result into the target database; wherein the second model context protocol server is connected to the target database.

2. The method according to claim 1, characterized in that, Sending a call request to the first model context protocol server to obtain the unstructured text data to be processed specifically includes: establishing a connection with the first model context protocol server and sending a file parsing request; the first model context protocol server connecting to the local or remote file system, reading and parsing the unstructured text data and returning it.

3. The method according to claim 1, characterized in that, The first processing module is configured as a metadata analysis definition agent; wherein, the construction process of the metadata analysis definition agent includes: Configure the first role definition prompt word, which is a user-defined prompt word for the scenario defined by metadata analysis. Construct a first task instruction to constrain the large language model to recognize entities, elements and relationships in the text; set a first output constraint to constrain the large language model to output metadata definitions in JSON format, wherein the metadata definitions include at least field names, field types, field lengths and field descriptions; The first role definition prompt and the first task instruction are combined to form the first system prompt.

4. The method according to claim 1, characterized in that, The second processing module is configured as a key element extraction agent; wherein, the construction process of the key element extraction agent includes: Configure a second role definition prompt word, which is a role prompt word for the user-defined metadata extraction information scenario; Construct a second task instruction, receive the metadata definition scheme as input constraints, and constrain the large language model to traverse the unstructured text data, match and extract specific numerical values ​​or text fragments corresponding to the metadata definition scheme, and output them in the form of key-value pairs; The second system prompt is formed by combining the second role definition prompt and the second task instruction.

5. The method according to claim 1, characterized in that, Before inputting the unstructured text data into the first processing module, the method further includes: The intent recognition agent is invoked to classify the unstructured text data and determine the business domain type of the data; Based on the business domain type, a corresponding industry standard knowledge base is matched as auxiliary input for the first processing module.

6. The method according to claim 1, characterized in that, Sending a call request to the second model context protocol server to write the structured data result into the target database specifically includes: establishing a connection with the second model context protocol server and sending a data write request; the second model context protocol server connecting to the target database and performing the storage operation of the structured data.

7. The method according to claim 1, characterized in that, The method further includes: After dynamically generating the metadata definition scheme, the format validity of the metadata definition scheme output by the first processing module is determined. If the determination is invalid, the exception handling module is called to assemble the exception information and terminate the current governance process. After generating the structured data results, the structured data results output by the second processing module are successfully extracted. If the extracted content does not conform to the metadata definition scheme, the exception handling module is called to assemble the exception information and terminate the current governance process. After storing the data in the target database, the system receives the write status returned by the database. If the write operation fails, the system calls the exception handling module to assemble the write exception information.

8. A data governance system based on a large-scale intelligent agent, characterized in that, The system includes: The first model context protocol server encapsulates file parsing tools and is configured to connect to local or remote file systems. The second model context protocol server encapsulates a database connection tool and is configured to connect to the target database. A Model Context Protocol (MGP) client is configured to communicate with the first MGP server and the second MGP server. The model context protocol client includes: The first processing module is configured to use a large language model to generate a metadata definition scheme that includes field names, field types and field descriptions based on the semantic content of the unstructured text data; The second processing module is configured to receive the unstructured text data and the metadata definition scheme as a constraint context; based on the metadata definition scheme, it maps and extracts the corresponding key element values ​​from the unstructured text data to generate structured data results.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images