A storage method and management system for highly correlated big data

Abstract

The invention discloses a storage method and management system for high-association big data. The management system includes a storage module and a unified data management module; the storage module includes an Hashmap model for storing data entity contents, a relation model for storing data entity attributes, and a graph data model for storing association relations among data entities; each data entity sets an entity type and a unique ID number, and an association relation between an attribute and a content of the same data entity is established through the ID number of the data entity; and the unified data management module is used for performing adding, delete, update, and inquiry on association relations, attributes, data contents of data entities in the storage module. The storage method and management system for the high-association big data can store and manage big data sets, and can support high-efficiency association inquiry analysis.

Description

technical field [0001] The invention belongs to the field of big data storage, and in particular relates to a storage method and management system for highly correlated big data. Background technique [0002] In the era of big data, enterprises or organizations pay more and more attention to the value of data, and gradually start the collection, storage, analysis and utilization of big data. In these large datasets, correlations between data are ubiquitous. Especially in application scenarios closely related to individual users, such as social network big data and medical big data, data objects are highly correlated. The complex links between the data in these highly correlated datasets often have huge analytical value. For example, the friendship between social users, the association between medicines and patients, and so on. At the same time, these highly correlated large data sets also have the characteristics of large scale, high speed, and diversity. Therefore, in or...

AI Technical Summary

Problems solved by technology

However, due to the lack of consideration of complex associations between data in highly correlated large data sets, and the lack of matching data models, storage methods, and query methods, these data sets stored in different databases or file systems are often independent of each other. A lot of redundancy, and it is also inefficient when performing complex relationship queries

[0006] In short, in the field of big data storage, there is still a lack of an efficient storage and management method for large data sets with a high degree of correlation, which cannot meet both the storage requirements of big data and the efficient analysis of complex relationships among them.

Images