Universal unstructured data index construction method and device and medium

By combining deep learning with large language models, we extract features and cluster unstructured data to generate a semantically consistent index structure, solving the versatility and efficiency issues of unstructured data management in existing technologies and achieving efficient and intelligent multimodal data retrieval.

CN120653807AActive Publication Date: 2025-09-16ZHEJIANG UNIV +1
6 Cites 1 Cited by

Patent Information

Application Number
CN202511161206.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-08-19
Publication Date
2025-09-16
Estimated Expiration
2045-08-19

AI Technical Summary

Technical Problem

Existing unstructured data management solutions lack versatility and are difficult to adapt to the management and retrieval needs of various data types. The retrieval method is mainly based on keywords, which makes it difficult to achieve efficient content-based and semantic-based retrieval. The index structure is inefficient and has low levels of automation and intelligence, and cannot meet the real-time or near-real-time business scenario requirements of large-scale data.

Method used

A deep learning model is used for feature extraction. A recursive clustering method based on automatic anchor point selection and hyperplane partitioning is used to perform unsupervised grouping of high-dimensional feature vectors to generate semantically consistent clusters. A large language model is used to automatically generate semantic summaries and generate independent index units for each cluster. It supports selection, multi-category combination and restricted queries to achieve automated screening.

Benefits of technology

It improves the efficiency and accuracy of unstructured data retrieval, reduces manual annotation costs, enhances the interpretability of retrieval results, adapts to multimodal data management, reduces system complexity and maintenance costs, and supports real-time business scenarios.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention discloses a universal unstructured data index construction method and device and a medium, and is suitable for multi-modal data such as pictures, videos and texts. The method comprises the steps that adaptive deep learning models are adopted for feature extraction for different modal data; building a multi-level cluster structure based on a recursive clustering method of automatic anchor point selection and hyperplane division; automatically generating a semantic abstract for each cluster by using a large language model; an index structure is efficiently organized through information such as categories, clusters and abstracts, and multi-dimensional retrieval is supported; and during query processing, a discrimination model is trained at low cost by using a very small amount of existing labels in the clustering clusters, so that efficient clustering cluster screening and multi-class combined query are realized, and a user does not need to specify positive and negative samples. The method has the advantages of being high in automation degree, low in labeling cost, high in expandability, capable of supporting multi-modal data, unique in labeling inheritance and the like, and the index construction and retrieval efficiency of large-scale unstructured data is remarkably improved.
Need to check novelty before this filing date? Find Prior Art