Universal data management system and method based on multi-modal intelligent agent

Abstract

The invention provides a universal data management system and method based on a multi-modal agent, electronic equipment, a storage medium and a program product. The universal data management method based on the multi-modal intelligent agent comprises the following steps: S1, defining and realizing data acquisition, data cleaning, data labeling and a data storage tool set; s2, designing a task Prompt and inputting the task Prompt into the multi-mode intelligent agent; and S3, the multi-modal agent generates a treatment plan according to the treatment task and requirements and comprehensively uses the tool chain to complete the data treatment task. Through the data governance system and the data governance method, the working process is simplified; each team only needs to pay attention to development and maintenance of own tools and does not need to pay attention to the complexity of the whole process, so that task decoupling is realized; the multi-modal intelligent agent can flexibly select and switch different tools, and the more the tools are, the stronger the problem solving capability of the system is; the method is suitable for various data governance scenes, is not limited by specific industries and applications, and has high universality.

Description

Technical Field

[0001] The present application relates to the field of information data processing technology, and in particular to a universal multimodal data governance system, method, electronic device, storage medium and program product based on a multimodal intelligent agent. Background Art

[0002] Existing data collection, cleaning, labeling and storage is a very discrete and customized process.

[0003] The data governance process is complex to design. The existing data governance process requires close collaboration between multiple teams, and the design and coordination work is complex and error-prone. The data governance framework has low flexibility and data governance tools are difficult to expand.

[0004] The data governance framework is fixed and difficult to adapt to different data governance needs. The tool scalability is poor and the cost of adding or replacing tools is high. It is impossible to flexibly organize each link, and each link has a strong dependency.

[0005] For example: to construct a data set of 100,000 car brand pictures, the data collection team first collects the car pictures, which requires the super-redundant collection of 1 million car pictures; then the pictures are cleaned, such as car target detection, watermark detection, and mosaic detection; the filtered pictures are handed over to the data annotation team for manual annotation, such as annotation of car brands, quality verification, etc.; the data is exported from the annotation system and handed over to the demander. Summary of the invention

[0006] The present invention provides a universal multimodal data governance system, method, electronic device, storage medium, and program product based on a multimodal intelligent agent, which flexibly organizes various links together so that personnel in each link only need to focus on developing and maintaining their own tools. The multimodal intelligent agent calls the tools as needed, and through the coordination and control of the intelligent agent, efficient management of data collection, cleaning, labeling, and storage is achieved.

[0007] The data collection team only needs to provide one or more image collection tools, and the multimodal agent will autonomously select the download tool for downloading.

[0008] Using the multimodal agent of the present application, during the data cleaning process, the agent independently selects the cleaning tool; the agent pushes the data that needs to be annotated to the annotation platform;

[0009] The intelligent agent can determine whether the data needs to be labeled, store the data that does not need to be labeled, and only give the data that needs to be labeled to downstream tasks.

[0010] In a first aspect, the present application provides a general multimodal data governance system based on a multimodal agent, including a multimodal agent, a data acquisition module, a data cleaning module, a data annotation module, and a data storage module.

[0011] The multimodal agent is the core of the entire system, responsible for coordinating and controlling the work of each module. The agent realizes efficient management of data governance by autonomously selecting and calling appropriate tools. The multimodal agent automatically completes data crawling, cleaning, labeling and storage tasks through task allocation and tool calling. The agent has the ability of self-learning and optimization, and can continuously improve data governance strategies based on historical data and task requirements.

[0012] Furthermore, the data acquisition module is responsible for acquiring raw data from various network resources. In some embodiments, the data acquisition module includes multiple acquisition tools, each of which focuses on a specific type of data acquisition task. The acquisition tools are developed by the data acquisition team and integrated into the system. The multimodal agent autonomously selects and calls the appropriate acquisition tool to download data according to the task requirements.

[0013] Furthermore, the data cleaning module is responsible for cleaning the collected raw data. In some embodiments, the module includes multiple cleaning tools, which are used for different types of cleaning tasks. The cleaning tools include object detection, watermark detection, and mosaic detection. The multimodal agent selects appropriate cleaning tools to process the data according to the data characteristics and cleaning requirements.

[0014] Furthermore, the data annotation module is responsible for manually or automatically annotating the cleaned data. In some embodiments, the annotation platform provides a user-friendly interface to facilitate the annotation personnel to annotate the data. It includes various annotation auxiliary tools, such as automatic label generation, quality inspection, etc. The multimodal agent is responsible for pushing the data to be annotated to the annotation platform and receiving the annotated data after the annotation is completed.

[0015] Furthermore, the data storage module is responsible for classifying and storing the cleaned and labeled data. This module ensures the security and traceability of the data. In some embodiments, the storage system includes a variety of data storage solutions, such as local storage, cloud storage, etc. The multimodal agent selects a suitable storage system for data storage according to the data characteristics and storage requirements.

[0016] In a second aspect, the present application provides a general multimodal data governance method based on a multimodal agent, comprising the following steps:

[0017] Step 1: Define and implement a set of tools for data collection, data cleaning, data annotation, and data storage;

[0018] Step 2: Design the task prompt and input it into the multimodal agent;

[0019] Step 3: The multimodal agent generates a governance plan based on the governance tasks and requirements and uses a comprehensive tool chain to complete the task.

[0020] In some embodiments, in step one, the definition tool format is as follows:

[0021] {"module":"module_name",

[0022] "name":"function_name",

[0023] "description":"Detailed description of the tool's functions",

[0024] "parameters":[{"name":"Parameter name",

[0025] "description":"Parameter meaning",

[0026] "type":"string",

[0027] "required":true}]}

[0028] In some embodiments, the collection tool set includes one or two or three or four of a picture collection tool, a video collection tool, an audio collection tool, and a text collection tool;

[0029] The cleaning tool set includes text cleaning tools, image cleaning tools, video cleaning tools, and audio cleaning tools.

[0030] Text cleaning tool, with functions such as advertisement cleaning, paragraph deduplication, and fluency recognition;

[0031] Image cleaning tool with functions such as watermark detection, object recognition, and OCR;

[0032] Video cleaning tools with quality detection, audio extraction and other functions;

[0033] Audio cleaning tool with quality detection, event detection, ASR and other functions;

[0034] Annotation toolset: text annotation, image annotation, video annotation, audio annotation, etc.

[0035] Storage toolset: local storage, cloud storage;

[0036] In step 2, you can design the following prompt.

[0037] Prompt example 1:

[0038] You are a multimodal agent, and your role is to use the tools I provide for data governance. In order to efficiently solve a data governance task, you need to use the tools in a reasonable combination.

[0039] The tools are as follows: {tool definition list}

[0040] Governance tasks: {present specific tasks}

[0041] Governance requirements: {present specific requirements}

[0042] Multi-modal intelligent output is as follows:

[0043] Step 1. Image collection.

[0044] Step 2. Watermark check.

[0045] Step 3: Target detection.

[0046] Step 4: Target object labeling.

[0047] Step 5. Store the data.

[0048] Prompt example 2:

[0049] You are a multimodal agent, and your role is to use the tools I provide for data governance. In order to efficiently solve a data governance task, you need to use the tools reasonably. Use the tools step by step, and you can determine which tools to use based on the data content. The tools are as follows: {Tool definition list}

[0050] Governance task: {Multimodal input} Different data types use different labels ,

[0051] <video> 、 <audio> 、 <text>(The difference from Prompt 1 is that you can enter data of different modes here)

[0052] Governance requirements: {Present specific requirements}

[0053] For example, governance tasks: / path / to / image.jpg Manage this image Management requirements: Check watermarks, whether there is a face, and mark the location of the target object

[0054] You can use the following prompt to submit to the multimodal agent. The agent uses the tool chain in combination to complete this task based on the governance tasks and requirements.

[0055] Multi-modal intelligent output:

[0056] There are no watermarks or faces in this picture (the multimodal large model can recognize them by itself). I only need to call the image_target_annotation tool to mark the location of the target objects in the picture.

[0057] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the general multimodal data governance method based on a multimodal agent as described in the second aspect above is implemented.

[0058] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, it implements the general multimodal data governance method based on a multimodal agent as described in the second aspect above.

[0059] In a fifth aspect, the present application provides a computer program product, including a computer program, characterized in that when the computer program is executed by a processor, it implements the general multimodal data governance method based on a multimodal agent as described in the second aspect above.

[0060] The present invention aims to build a general multimodal data governance system based on multimodal agents through a more efficient and flexible method. The advantage is that through the coordination of multimodal agents, the automation and intelligence of each link can be realized, human intervention can be reduced, and the workflow can be simplified; each team only needs to focus on developing and maintaining its own tools without worrying about the complexity of the entire process, thus achieving task decoupling;

[0061] Multimodal agents can flexibly select and switch different tools. The more tools there are, the stronger the problem-solving ability of the system. It is applicable to various data governance scenarios, not limited by specific industries and applications, and has strong versatility. This method can save steps. For example, multimodal agents can directly determine whether the image has a watermark, while the large language model can only be handed over to the downstream tool-watermark inspection model. The data governance framework of this data governance system is highly flexible, and the expansion of data governance tools is relatively simple. For example, another task is added to collect 50,000 plant photos and classify plant species, requiring the images to be watermark-free. Since the data governance system has already collected car pictures and has the ability to collect pictures and detect picture watermarks, it only needs to develop a plant variety classification program, that is, an annotation tool. Traditionally, even with tool reuse, the program will increase coupling. If it is implemented using a multimodal agent, it only needs to implement the plant variety classification, and other tool calls are completed by the agent, which expands the plant variety annotation tool and can quickly meet the collection needs of 50,000 plants. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0063] Figure 1 It is a structural diagram of a general multimodal data governance system based on a multimodal agent provided in an embodiment of the present application;

[0064] Figure 2 It is a flowchart of a general multimodal data governance method based on a multimodal agent provided in an embodiment of the present application;

[0065] Figure 3 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application; DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0067] Figure 1 It is a structural diagram of a general multimodal data governance system based on a multimodal agent provided in an embodiment of the present application.

[0068] The general multimodal data governance system based on multimodal agent includes a multimodal agent, a data acquisition module, a data cleaning module, a data annotation module, and a data storage module.

[0069] Furthermore, the multimodal agent automatically completes data crawling, cleaning, labeling and storage tasks through task allocation and tool calling. The agent has the ability of self-learning and optimization, and can continuously improve data governance strategies based on historical data and task requirements.

[0070] Furthermore, the data collection module is responsible for obtaining raw data from various network resources. The data collection module includes multiple collection tools, each of which focuses on a specific type of data collection task. The collection tools are developed by the data collection team and integrated into the system. The multimodal agent autonomously selects and calls the appropriate collection tool to download data according to the task requirements.

[0071] Furthermore, the data cleaning module is responsible for cleaning the collected raw data. This module includes multiple cleaning tools, which are used for different types of cleaning tasks. Cleaning tools include object detection, watermark detection, and mosaic detection. The multimodal agent selects appropriate cleaning tools to process the data according to the data characteristics and cleaning requirements.

[0072] Furthermore, the data annotation module is responsible for manually or automatically annotating the cleaned data. The annotation platform provides a user-friendly interface to facilitate the annotation personnel to annotate the data. It includes various annotation auxiliary tools, such as automatic label generation, quality inspection, etc. The multimodal agent is responsible for pushing the data to be annotated to the annotation platform and receiving the annotated data after the annotation is completed.

[0073] Furthermore, the data storage module is responsible for classifying and storing the cleaned and labeled data. This module ensures the security and traceability of the data. The storage system includes a variety of data storage solutions, such as local storage, cloud storage, etc. The multimodal agent selects a suitable storage system for data storage based on the data characteristics and storage requirements.

[0074] Figure 2 This is Example 1 of a general multimodal data governance method based on a multimodal agent provided in an embodiment of the present application.

[0075] Step 1: Define and implement the toolset used for data governance;

[0076] Specifically, the definition tool format is as follows:

[0077] {"module":"module_name",

[0078] "name":"function_name",

[0079] "description":"Detailed description of the tool's functions",

[0080] "parameters":[{"name":"Parameter name",

[0081] "description":"Parameter meaning",

[0082] "type":"string",

[0083] "required":true}]};

[0084] Specifically, the toolset includes a collection toolset, a cleaning toolset, a labeling toolset, and a storage toolset.

[0085] Taking the collection toolset as an example, the collection toolset is defined as follows:

[0086] {"module":"Collection toolset",

[0087] "name":"image_downloader",

[0088] "description":"Image download tool, accepts a keyword parameter, searches for the keyword and downloads related images.",

[0089] "parameters":[{"name":"keyword",

[0090] "description":"Keywords",

[0091] "type":"string",

[0092] "required":true}];

[0093] Taking the cleaning toolset as an example, the cleaning toolset is defined as follows:

[0094] {"module":"Cleaning tool set",

[0095] "name":"image_watermark_detection",

[0096] "description":"Image watermark check tool, check watermark based on the given image.","parameters":[{"name":"image_path",

[0097] "description":"Image address",

[0098] "type":"string",

[0099] "required":true}]}

[0100] Taking the annotation toolset as an example, the definition of the annotation toolset is as follows:

[0101] {"module":"Annotation Toolset",

[0102] "name":"image_car_annotation",

[0103] "description":"Image car position detection tool, marking car position",

[0104] "parameters":[{"name":"image_path",

[0105] "description":"Image address",

[0106] "type":"string",

[0107] "required":true}]}

[0108] Step 2: Design the task prompt and input it into the multimodal agent.

[0109] The specific examples are, but not limited to, the following.

[0110] You are a multimodal agent, and your role is to use the tools I provide for data governance. In order to efficiently solve a data governance task, you need to use the tools reasonably. Use the tools step by step, and you can determine which tools to use based on the data content. The tools are as follows: {Tool definition list}

[0111] Governance tasks: {presented as specific tasks}

[0112] Governance requirements: {presented as specific requirements}

[0113] To collect the target object pictures, the prompts input to the multimodal agent are as follows:

[0114] You are a multimodal agent, and your role is to use the tools I provide for data governance. In order to efficiently solve a data governance task, you need to use the tools reasonably. Use the tools step by step, and you can determine which tools to use based on the data content. The tools are as follows:

[0115] {\"module\":\"Collection toolset

[0116] \",\"name\":\"image_downloader\",\"description\":\"Image download tool, accepts a keyword parameter, searches for the keyword and downloads related images.

[0117] \",\"parameters\":[{\"name\":\"keyword\",\"description\":

[0118] \"keyword\",\"type\":\"string\",\"required\":true}]}

[0119] {\"module\":\"Cleaning tools set

[0120] \",\"name\":\"image_watermark_detection\",\"description\":\"Image watermark detection tool, performs watermark detection based on a given image.

[0121] \",\"parameters\":[{\"name\":\"image_path\",\"description\":

[0122] \"Image address\",\"type\":\"string\",\"required\":true}]}

[0123] {\"module\":\"Annotation toolset

[0124] \",\"name\":\"image_face_detection\",\"description\":\"Image face detection tool, performs face detection based on a given image.

[0125] \",\"parameters\":[{\"name\":\"image_path\",\"description\":

[0126] \"Image address\",\"type\":\"string\",\"required\":true}]}

[0127] {\"module\":\"Annotation toolset

[0128] \",\"name\":\"image_car_annotation\",\"description\":\"Image object position detection tool, marking the target position

[0129] \",\"parameters\":[{\"name\":\"image_path\",\"description\":

[0130] \"Image address\",\"type\":\"string\",\"required\":true}]}

[0131] {\"module\":\"Storage toolset

[0132] \",\"name\":\"local_storage\",\"description\":\"Storage data

[0133] \",\"parameters\":[{\"name\":\"file_path\",\"description\":

[0134] \"File address

[0135] \",\"type\":\"string\",\"required\":false},{\"name\":\"text\",\"description\":\"text content

[0136] \",\"type\":\"string\",\"required\":false}]}

[0137] Governance task: Collect 100,000 pictures of target brands

[0138] Governance requirements: no watermark, no face, the location of the target object needs to be marked

[0139] The target objects mentioned above may be automobiles, welds in welding, or medical devices. Of course, they are not limited to automobiles, welds, or medical devices, and may also be other target objects.

[0140] Step 3: Multimodal Agent Output Data Governance Method

[0141] Specifically,

[0142] Image collection: Use the image_downloader tool to collect images of the target object.

[0143] Watermark detection: Use the image_watermark_detection tool to detect and exclude images with watermarks.

[0144] Face detection: Use the image_face_detection tool to detect and exclude images with faces.

[0145] Target object position annotation: Use the image_car_annotation tool to annotate the position of the target object in the image.

[0146] Store data: Use the local_storage tool to store the processed images locally.

[0147] Stated below is Example 2 of a general multimodal data governance method based on a multimodal agent provided by the present application.

[0148] Step 1: Define and implement a set of tools for data collection, data cleaning, data labeling, and data storage.

[0149] The specific definition method can be the same as the embodiment 1 of the general multimodal data governance method based on multimodal intelligent agent, or other definition methods can be used.

[0150] Step 2: Design the task prompt and input it into the multimodal agent.

[0151] You are a multimodal agent, and your role is to use the tools I provide for data governance. In order to efficiently solve a data governance task, you need to use the tools reasonably. Use the tools step by step, and you can determine which tools to use based on the data content. The tools are as follows: {Tool definition list}

[0152] Governance task: {Multimodal input} Different data types use different labels ,

[0153] <video> 、 <audio> 、 <text>

[0154] Governance requirements: {presented as specific requirements}

[0155] For example,

[0156] Governance tasks: / path / to / image.jpg Manage this picture

[0157] Governance requirements: Check watermarks, check for faces, and mark the location of the target object

[0158] You can use the following prompt to submit to the multimodal agent. The agent uses the tool chain in combination to complete this task based on the governance tasks and requirements.

[0159] The main difference between the second embodiment and the first embodiment lies in the Prompt. The difference between the Prompt in the second embodiment and the Prompt in the first embodiment is that the Prompt in the second embodiment can input data in different modes.

[0160] In the second embodiment, the specific input prompt is as follows:

[0161] You are a multimodal agent, and your role is to use the tools I provide for data governance. In order to efficiently solve a data governance task, you need to use the tools reasonably. Use the tools step by step, and you can determine which tools to use based on the data content. The tools are as follows:

[0162] {\"module\":\"Collection toolset

[0163] \",\"name\":\"image_downloader\",\"description\":\"Image download tool, accepts a keyword parameter, searches for the keyword and downloads related images.

[0164] \",\"parameters\":[{\"name\":\"keyword\",\"description\":

[0165] \"keyword\",\"type\":\"string\",\"required\":true}]}

[0166] {\"module\":\"Cleaning tools set

[0167] \",\"name\":\"image_watermark_detection\",\"description\":\"Image watermark detection tool, performs watermark detection based on a given image.

[0168] \",\"parameters\":[{\"name\":\"image_path\",\"description\":

[0169] \"Image address\",\"type\":\"string\",\"required\":true}]}

[0170] {\"module\":\"Annotation toolset

[0171] \",\"name\":\"image_face_detection\",\"description\":\"Image face detection tool, performs face detection based on a given image.

[0172] \",\"parameters\":[{\"name\":\"image_path\",\"description\":

[0173] \"Image address\",\"type\":\"string\",\"required\":true}]}

[0174] {\"module\":\"Annotation toolset

[0175] \",\"name\":\"image_car_annotation\",\"description\":\"Image car position detection tool, annotate car position

[0176] \",\"parameters\":[{\"name\":\"image_path\",\"description\":

[0177] \"Image address\",\"type\":\"string\",\"required\":true}]}

[0178] {\"module\":\"Storage toolset

[0179] \",\"name\":\"local_storage\",\"description\":\"Storage data

[0180] \",\"parameters\":[{\"name\":\"file_path\",\"description\":

[0181] \"File address

[0182] \",\"type\":\"string\",\"required\":false},{\"name\":\"text\",\"description\":\"text content

[0183] \",\"type\":\"string\",\"required\":false}]}

[0184] Governance tasks: / path / to / image.jpg Manage this image Management requirements: check watermarks, whether there is a face, and mark the location of the target object.

[0185] Step 3: Output of multimodal agent.

[0186] There is no watermark or face in this picture. I just need to call the image_car_annotation tool to mark the location of the target object in the picture.

[0187] It is important to note that the multimodal large model can recognize by itself that there are no watermarks or faces in the image.

[0188] Figure 3 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 3 As shown, the electronic device may include: a processor 301 (processor), a communication interface 302 (Communications Interface), a memory 303 (memory) and a communication bus 304, wherein the processor, the communication interface and the memory communicate with each other through the communication bus. The processor can call the logic instructions in the memory to execute the data governance method, which includes: S1: define and implement data collection, data cleaning, data annotation, and data storage tool sets; S2: design task prompts and input them into the multimodal agent; S3: the multimodal agent generates a governance plan based on the governance tasks and requirements and uses the tool chain in combination to complete the data governance tasks. Specifically, the processor can be a central processing unit (CPU), a dedicated integrated circuit

[0189] The processor may be an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or a Complex Programmable Logic Device (CPLD). The processor may also adopt a multi-core architecture.

[0190] The logical instructions in the memory can be implemented in the form of software functional units and can be stored in a processor-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program code. In some embodiments, a non-transitory computer-readable storage medium is also provided, on which a computer program is stored, characterized in that when the computer program is executed by a processor, the following data governance method is implemented, the method comprising: S1: defining and implementing a set of data collection, data cleaning, data labeling, and data storage tools; S2: designing task prompts and inputting them into a multimodal intelligent agent; S3: the multimodal intelligent agent generates a governance plan based on governance tasks and requirements and uses a comprehensive tool chain to complete data governance tasks.

[0191] In some embodiments, a computer program product is also provided, including a computer program, characterized in that when the computer program is executed by a processor, the following data governance method is implemented, the method comprising: S1: defining and implementing a set of data collection, data cleaning, data annotation, and data storage tools; S2: designing a task prompt and inputting it into a multimodal intelligent agent; S3: the multimodal intelligent agent generates a governance plan based on governance tasks and requirements and uses a comprehensive tool chain to complete data governance tasks.

[0192] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program codes. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, may be implemented by computer-executable instructions. These computer-executable instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the functions in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These processor-executable instructions may also be stored in a processor-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the processor-readable memory produce a manufactured product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These processor executable instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are performed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps of the functions specified in one or more boxes. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.< / text> < / audio> < / video> < / text> < / audio> < / video>

Claims

1. A data governance system based on multimodal agents, characterized by Including multimodal intelligent agents, data collection tools, data cleaning tools, data labeling tools, and data storage tools. The multimodal intelligent agents independently select and call data collection tools, data cleaning tools, data labeling tools, and data storage tools.

2. According to the multimodal agent-based data governance system according to claim 1, its data collection tools include one, two, three or four of picture collection tools, video collection tools, audio collection tools and text collection tools.

3. A data governance method based on a multimodal agent, the method comprising the following steps: S1: Define and implement data collection, data cleaning, data labeling, and data storage tool set; S2: Design task prompts and input them into the multimodal agent; S3: The multimodal agent generates a governance plan based on governance tasks and requirements and uses a comprehensive tool chain to complete data governance tasks.

4. According to the multimodal agent-based data governance method of claim 3, in step S1, the tool format is defined as follows: {"module":"module_name", "name":"function_name", "description":"Detailed description of the tool's functions", "parameters":[{"name":"Parameter name", "description":"Parameter meaning", "type":"string", "required":true}]}.

5. According to claim 3, the multi-agent based data governance method is used to collect data sets of any target object.

6. According to the multi-agent based data management method described in claim 3, the method is used to manage any image, to check whether the image has a watermark, whether there is a face, and to mark the location of the target object.

7. The multi-agent-based data governance method according to claim 5 is used to collect data sets of automobiles, plants, or medical devices.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it implements the multimodal agent-based data governance method as described in any one of claims 3 to 7.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the multimodal agent-based data governance method as described in any one of claims 3 to 7.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the multimodal agent-based data governance method as described in any one of claims 3 to 7.

Images