Heterogeneous data semantic metadata discovery method and system based on big and small model collaboration

Abstract

The invention discloses a heterogeneous data semantic metadata discovery method and system based on big and small model collaboration, and the method comprises the steps: configuring data source information in a visual Web page according to the needs of a service, reading the data source information through a multi-modal data interface, and receiving the original data of the service; according to the type of the original data, automatically selecting and calling a large model and a special small model to perform semantic analysis on the original data, and generating basic semantic metadata of heterogeneous data resources; generating extended semantic metadata; automatically checking the extracted semantic metadata through a metadata quality evaluation module; establishing a dynamic index relationship between the verified semantic metadata and the original data; the system comprises a data access module, a data generation module, a quality evaluation module and a dynamic index module. According to the method, the extracted semantic metadata can be automatically evaluated, the persistence of the mapping relation can be realized, and a user can conveniently mine and utilize data resources.

Description

Technical Field

[0001] This invention relates to semantic metadata discovery technology, and more particularly to a method and system for discovering semantic metadata of heterogeneous data based on size model collaboration. Background Technology

[0002] With the rise of big data technology, the amount of data collected by business systems is increasing dramatically. The diversity and complexity of data resources have significantly increased, making it difficult to effectively understand, discover, and utilize data. While using semantic metadata to describe data resources is an effective solution, the collected data resources are often difficult to understand and discover due to their complex structures. Currently, describing data resources using semantic metadata is a common method, but most semantic metadata is currently registered and labeled manually by data administrators or business experts. This method has significant drawbacks: it is labor-intensive, impractical for registering each massive data asset individually; it requires a high level of expertise, demanding a deep understanding of both business and data, leading to a shortage of such personnel; updates are delayed, as metadata cannot be updated in a timely manner after changes in data structure or business meaning, making consistency difficult to guarantee; different personnel may have differing understandings of the same data, affecting the full realization of data value. Due to missing or incomplete semantic metadata, data consumers struggle to quickly find the data they need and understand its underlying meaning and credibility, thus limiting data sharing, reuse, and value extraction. Therefore, it is necessary to propose a heterogeneous data semantic metadata discovery method to address these existing problems. Summary of the Invention

[0003] Purpose of the invention: The purpose of this invention is to provide a method and system for discovering semantic metadata of heterogeneous data based on size model collaboration.

[0004] Technical solution: The heterogeneous data semantic metadata discovery method based on size model collaboration described in this invention includes the following steps: Step 1: Access raw business data. Configure data source information in a visual web page according to business needs, and read data source information through a multimodal data interface to receive raw business data. Step 2: Basic semantic metadata generation. Based on the type of the original data, the system automatically selects and calls the large model and the dedicated small model to perform semantic parsing on the original data, generating basic semantic metadata for heterogeneous data resources. Step 3: Extended semantic metadata generation. Based on the dynamic knowledge graph, the initial semantic tags are subjected to contextual reasoning to generate extended semantic metadata. Step 4: Metadata quality assessment, which automatically verifies the extracted semantic metadata through the metadata quality assessment module; Step 5: Persist the index relationship. Establish a dynamic index relationship between the verified semantic metadata and the original data, and store it in the metadata database.

[0005] Furthermore, the original data types of the business in step 1 include structured data and unstructured data; the multimodal data interface includes a database access unit and a file data access unit.

[0006] Furthermore, the database access unit supports access to database connection information, including the database's network address, port, and database instance.

[0007] Furthermore, the file data access unit supports access to the connection information of the file data source, including the file's network address, port, and path information.

[0008] Further, step 2 includes: Step 2-1: Determine the data type. If it is a database table, query the information of the corresponding table through the database connection. Combine the information of the table with the prompt words and input them into the language model to obtain the data subject, field information, database tag set and semantic metadata of the data summary of the database table. Step 2-2: If it is a document type, combine the prompt words and use the language model to extract the document's summary and tags, and generate a description of the document's topic; Steps 2-3: If it is an image, combine the prompt words, call the visual big model to identify the target type in the image, then call the corresponding target recognition model to perform professional analysis of the target in the image, generate the corresponding tag set, combine the prompt words and tag set, and then call the big model to generate image theme description and image content summary. Steps 2-4: If the video is a video file, extract the audio information from the video file. Steps 2-5: If it is an audio type, combine the prompt words, call the audio big model, convert the audio into text, and extract the summary and keywords as a tag set. Using the summary and tag set, use the big language model to generate the topic description of the audio.

[0009] Further, steps 2-4 include: Step 2-4-1: Extract keyframes from the target video using a convolutional neural network (CNN) or a frame differencing method based on OpenCV. Sequentially call a large visual model to identify the category of the keyframes, and then call the corresponding model to parse the corresponding image content to generate a label set. Step 2-4-2: By decoding the video, the audio file is separated, and then combined with the prompt words, the audio model is called to convert the audio into text, and the summary and keywords are extracted as tags; Step 2-4-3: Combine the tag set extracted in Step 2-4-1 and the summary extracted in Step 2-4-2 with prompt words to generate a summary of the video content and a description of the video's theme for the large language model.

[0010] Further, step 3 includes: Step 3-1: Combine the tag set in the basic semantic metadata of database, document, image, video and audio data resources with prompt words, and generate the corresponding dynamic knowledge graph query statement through a large language model; Step 3-2: Use query statements to query and access the corresponding associated tag entities in the knowledge graph, as extended semantic metadata of the corresponding data resources.

[0011] Further, step 4 includes: Step 4-1: Combining the prompt words, classify and identify the generated semantic metadata items using a large language model; Step 4-2: If all semantic metadata items of the data resource belong to the same category, then there is no problem; Step 4-3: If the semantic metadata items of the data resource have multiple categories, the category with the most percentages is regarded as the type of semantic metadata of the data resource, and the others are marked as items to be confirmed.

[0012] Furthermore, the dynamic index relationship established between the semantic metadata verified in step 5 and the original data is a mapping relationship between the two.

[0013] The heterogeneous data semantic metadata discovery system based on size model collaboration described in this invention includes: The data access module is used to configure data source information in a visual web page according to business needs, read data source information through a multimodal data interface, and receive raw business data. The data generation module is used to automatically select and call large models and dedicated small models to perform semantic parsing on the original data according to the type of the original data, and generate basic semantic metadata of heterogeneous data resources; and to perform context association reasoning on the preliminary semantic tags based on dynamic knowledge graphs to generate extended semantic metadata. The quality assessment module is used to automatically verify the extracted semantic metadata through the metadata quality assessment module; The dynamic indexing module is used to establish a dynamic index relationship between the verified semantic metadata and the original data, and store it in the metadata database.

[0014] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: The present invention has the function of automatically extracting semantic metadata of various data resources such as database data, document data, image data, video data and audio data through large models and domain-specific models. At the same time, the semantic metadata is extended through dynamic graphs. Finally, it can realize automatic evaluation and persistence of mapping relationships of the extracted semantic metadata, which facilitates users to mine and utilize data resources. Attached Figure Description

[0015] Figure 1 This is a flowchart of the present invention; Figure 2 The diagram illustrates the semantic metadata structure of various heterogeneous data sources, where (a) represents the semantic metadata of database tables, (b) represents the semantic metadata of documents, (c) represents the semantic metadata of images, (d) represents the semantic metadata of videos, and (e) represents the semantic metadata of audio. Figure 3 A schematic diagram illustrating the process of extracting basic semantic metadata from heterogeneous data resources; Figure 4 A schematic diagram illustrating the process of extracting semantic metadata for heterogeneous data resources. Detailed Implementation

[0016] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0017] like Figure 1 As shown, the heterogeneous data semantic metadata discovery method based on size model collaboration described in this invention includes the following steps: Step 1: Based on the needs of business metadata extraction, access various original business data resources by inputting data source parameter information on the visual web page. In this embodiment, if the business needs to extract semantic metadata of a database type, the configuration information of the data source includes the database type (such as Oracle, DM, Kingbase, Hasee, MySQL, etc.), the database's network address, port, and other information.

[0018] The original data types for business operations include structured and unstructured data. The multimodal data interface includes a database access unit and a file data access unit. The database access unit supports access to database connection information, including the database network address, port, database instance, etc., selects the range of database tables to be accessed, and collects information such as table comments and field names and field comments of the data tables. Supported database types include Oracle, DM, Kingbase, Hasee, MySQL, etc. The file data access unit supports access to file data source connection information, including the file's network address, port, path, and other information. Supported file data sources include FTP, MinIO, HDFS, etc. Step 2: In the basic semantic metadata generation module, the type of the incoming raw data is identified, and the large model and the dedicated small model are automatically selected to perform semantic parsing on the raw data, generating basic semantic metadata for heterogeneous data resources, such as... Figure 2 As shown, the present invention defines the structure of semantic metadata for various data resources such as database tables, documents, images, videos, and audio, wherein (a) is the semantic metadata of database tables, (b) is the semantic metadata of documents, (c) is the semantic metadata of images, (d) is the semantic metadata of videos, and (e) is the semantic metadata of audio.

[0019] like Figure 3 As shown, this invention defines a process for automatically extracting basic semantic metadata from various data resources. The specific steps include: Determine the data type. If it is a database table, query the corresponding table name, table comments, field names, field types, field comments, etc. through the database connection. Combine this information with the prompt words and input them into the language model to obtain semantic metadata such as the data theme, field information, database tag set, and data summary of the database table. If it is a document type, combined with prompt words, such as "extract the document's summary and top 10 keywords, and generate the document's topic", the language big model is used to extract the document's summary and keywords, and the keywords are used as the document's tags to generate a description of the document's topic; If it is an image, combined with prompts such as "identify which field this image belongs to", the visual big data model is called to identify the type of target appearing in the image (identify which professional field the image content belongs to). Then, the target recognition model of the corresponding professional field (such as the Yolo series) is called to perform professional analysis of the target appearing in the image, and generate a set of professional tags for the corresponding field. Combined with prompts (such as "generate the theme description and content summary of this image") and the tag set, the visual big data model (such as QWenVL) is called to generate the image theme description and image content summary. If the video is a video file, the audio information is extracted as follows: Keyframes are extracted from the target video using a Convolutional Neural Network (CNN) or a frame differencing method based on OpenCV. Combined with prompts such as "identify which domain this image belongs to," the keyframes are sequentially processed by a large-scale visual model (e.g., QWenVL) to identify the domain category. Then, the corresponding domain model is called to parse the image content and generate a domain-specific tag set. The video is decoded to separate the audio file. Then, combined with prompts such as "convert this audio file into text and extract the summary and the first 10 keywords," a large-scale audio model (e.g., QWen2-Audio) is called to convert the audio into text and extract the summary and keywords as tags. The extracted tag set and extracted summary, combined with prompt words, are used to generate a summary of the video content and a thematic description of the video for the large language model; If it is an audio file, the system uses prompts such as "extract the text information of this audio file" to call a large audio model (such as QWen2-Audio) to convert the audio into text. Then, using prompts, the system extracts the summary and keywords of the text and uses the summary and keywords as a tag set. Based on the summary and tag set, the system uses a large language model to generate a topic description of the audio.

[0020] Step 3: In the extended semantic metadata generation module, contextual reasoning is performed on the initial semantic tags based on the dynamic knowledge graph to generate extended semantic metadata, such as... Figure 4 As shown, by using the tag set in the basic semantic metadata of various data resources and using prompt words, such as "generate a graph 1-hop association query statement for a certain entity tag", the generated statement queries the corresponding graph and generates the corresponding extended semantic metadata. The tag set in the basic semantic metadata of database, document, image, video and audio data resources is combined with prompt words, such as "generate a knowledge graph query statement related to tag 1 and tag 2", and the corresponding dynamic knowledge graph query statement is generated through a large language model, such as "Match (XX), return XXX".

[0021] Use query statements to query and access the corresponding associated tag entities in the knowledge graph, which serve as extended semantic metadata for the corresponding data resources.

[0022] Step 4: In the metadata quality assessment module, the extracted semantic metadata is automatically classified by calling the large language model, and unqualified semantic metadata items are identified by judging the type of each semantic metadata. Combining prompts such as "identify the type category to which a semantic metadata item belongs", the generated semantic metadata items, including information such as data topic, summary, tags and related tags, are classified and identified using a large language model; If all semantic metadata items of the data resource belong to the same category, then there is no problem. If the semantic metadata items of the data resource have multiple categories, the category with the largest proportion is the type of semantic metadata of the data resource, and the others are marked as items to be confirmed.

[0023] Step 5: In the index relationship persistence module, a dynamic index relationship is established between the verified semantic metadata and the original data, and stored in the metadata database to support the subsequent application of semantic metadata; specifically, this includes establishing a mapping relationship between the semantic metadata item and the original data to support the subsequent retrieval and discovery of data.

[0024] The heterogeneous data semantic metadata discovery system based on size model collaboration described in this invention includes: The data access module is used to configure data source information in a visual web page according to business needs, read data source information through a multimodal data interface, and receive raw business data. The data generation module is used to automatically select and call large models and dedicated small models to perform semantic parsing on the original data according to the type of the original data, and generate basic semantic metadata of heterogeneous data resources; and to perform context association reasoning on the preliminary semantic tags based on dynamic knowledge graphs to generate extended semantic metadata. The quality assessment module is used to automatically verify the extracted semantic metadata through the metadata quality assessment module; The dynamic indexing module is used to establish a dynamic index relationship between the verified semantic metadata and the original data, and store it in the metadata database.

Claims

1. A method for discovering semantic metadata of heterogeneous data based on size-model collaboration, characterized in that, Includes the following steps: Step 1: Access raw business data. Configure data source information in a visual web page according to business needs, and read data source information through a multimodal data interface to receive raw business data. Step 2: Basic semantic metadata generation. Based on the type of the original data, the system automatically selects and calls the large model and the dedicated small model to perform semantic parsing on the original data, generating basic semantic metadata for heterogeneous data resources. Step 3: Extended semantic metadata generation. Based on the dynamic knowledge graph, the initial semantic tags are subjected to contextual reasoning to generate extended semantic metadata. Step 4: Metadata quality assessment, which automatically verifies the extracted semantic metadata through the metadata quality assessment module; Step 5: Persist the index relationship. Establish a dynamic index relationship between the verified semantic metadata and the original data, and store it in the metadata database.

2. The heterogeneous data semantic metadata discovery method based on size model collaboration according to claim 1, characterized in that, The original data types of the business in step 1 include structured data and unstructured data; the multimodal data interface includes a database access unit and a file data access unit.

3. The heterogeneous data semantic metadata discovery method based on size model collaboration according to claim 2, characterized in that, The database access unit supports access to database connection information, including the database's network address, port, and database instance.

4. The heterogeneous data semantic metadata discovery method based on size model collaboration according to claim 2, characterized in that, The file data access unit supports access to the connection information of the file data source, including the file's network address, port, and path information.

5. The heterogeneous data semantic metadata discovery method based on size model collaboration according to claim 1, characterized in that, Step 2 includes: Step 2-1: Determine the data type. If it is a database table, query the information of the corresponding table through the database connection. Combine the information of the table with the prompt words and input them into the language model to obtain the data subject, field information, database tag set and semantic metadata of the data summary of the database table. Step 2-2: If it is a document type, combine the prompt words and use the language model to extract the document's summary and tags, and generate a description of the document's topic; Steps 2-3: If it is an image, combine the prompt words, call the visual big model to identify the target type in the image, then call the corresponding target recognition model to perform professional analysis of the target in the image, generate the corresponding tag set, combine the prompt words and tag set, and then call the big model to generate image theme description and image content summary. Steps 2-4: If the video is a video file, extract the audio information from the video file. Steps 2-5: If it is an audio type, combine the prompt words, call the audio big model, convert the audio into text, and extract the summary and keywords as a tag set. Using the summary and tag set, use the big language model to generate the topic description of the audio.

6. The heterogeneous data semantic metadata discovery method based on size model collaboration according to claim 5, characterized in that, Steps 2-4 include: Step 2-4-1: Extract keyframes from the target video using a convolutional neural network (CNN) or a frame differencing method based on OpenCV. Sequentially call a large visual model to identify the category of the keyframes, and then call the corresponding model to parse the corresponding image content to generate a label set. Step 2-4-2: By decoding the video, the audio file is separated, and then combined with the prompt words, the audio model is called to convert the audio into text, and the summary and keywords are extracted as tags; Step 2-4-3: Combine the tag set extracted in Step 2-4-1 and the summary extracted in Step 2-4-2 with prompt words to generate a summary of the video content and a description of the video's theme for the large language model.

7. The heterogeneous data semantic metadata discovery method based on size model collaboration according to claim 1, characterized in that, Step 3 includes: Step 3-1: Combine the tag set in the basic semantic metadata of database, document, image, video and audio data resources with prompt words, and generate the corresponding dynamic knowledge graph query statement through a large language model; Step 3-2: Use query statements to query and access the corresponding associated tag entities in the knowledge graph, as extended semantic metadata of the corresponding data resources.

8. The heterogeneous data semantic metadata discovery method based on size model collaboration according to claim 1, characterized in that, Step 4 includes: Step 4-1: Combining the prompt words, classify and identify the generated semantic metadata items using a large language model; Step 4-2: If all semantic metadata items of the data resource belong to the same category, then there is no problem; Step 4-3: If the semantic metadata items of the data resource have multiple categories, the category with the most percentages is regarded as the type of semantic metadata of the data resource, and the others are marked as items to be confirmed.

9. The heterogeneous data semantic metadata discovery method based on size model collaboration according to claim 1, characterized in that, The dynamic index relationship established between the semantic metadata verified in step 5 and the original data is a mapping relationship between the two.

10. A heterogeneous data semantic metadata discovery system based on big-small model collaboration, implemented using the heterogeneous data semantic metadata discovery method based on big-small model collaboration as described in any one of claims 1-9, characterized in that, include: The data access module is used to configure data source information in a visual web page according to business needs, read data source information through a multimodal data interface, and receive raw business data. The data generation module is used to automatically select and call large models and dedicated small models to perform semantic parsing on the original data according to the type of the original data, and generate basic semantic metadata of heterogeneous data resources; and to perform context association reasoning on the preliminary semantic tags based on dynamic knowledge graphs to generate extended semantic metadata. The quality assessment module is used to automatically verify the extracted semantic metadata through the metadata quality assessment module; The dynamic indexing module is used to establish a dynamic index relationship between the verified semantic metadata and the original data, and store it in the metadata database.

Images