A storage method and management system for highly correlated big data

A technology of big data and association relationship, which is applied in the field of big data storage, can solve the problems of inefficient query, inability to meet the needs of big data storage and efficient analysis at the same time, and achieve the effect of improving query efficiency

CN106227800BActive Publication Date: 2020-02-21INST OF SOFTWARE - CHINESE ACAD OF SCI
7 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Publication Date
2020-02-21

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a storage method and management system for high-association big data. The management system includes a storage module and a unified data management module; the storage module includes an Hashmap model for storing data entity contents, a relation model for storing data entity attributes, and a graph data model for storing association relations among data entities; each data entity sets an entity type and a unique ID number, and an association relation between an attribute and a content of the same data entity is established through the ID number of the data entity; and the unified data management module is used for performing adding, delete, update, and inquiry on association relations, attributes, data contents of data entities in the storage module. The storage method and management system for the high-association big data can store and manage big data sets, and can support high-efficiency association inquiry analysis.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention belongs to the field of big data storage, and in particular relates to a storage method and management system for highly correlated big data. Background technique

[0002] In the era of big data, enterprises or organizations pay more and more attention to the value of data, and gradually start the collection, storage, analysis and utilization of big data. In these large datasets, correlations between data are ubiquitous. Especially in application scenarios closely related to individual users, such as social network big data and medical big data, data objects are highly correlated. The complex links between the data in these highly correlated datasets often have huge analytical value. For example, the friendship between social users, the association between medicines and patients, and so on. At the same time, these highly correlated large data sets also have the characteristics of large scale, high speed, and diversity. Therefore, in or...

Examples

Embodiment Construction

[0052] The key technologies and method implementations in the summary of the invention will be exemplarily explained below, but the scope of the invention will not be limited by such explanation.

[0053] 1) Dataset

[0054] Taking the data of a social networking site as an example, the data mainly includes user information data and Weibo information data. User information data includes user account, gender, age, hobbies, registration time, a list of other users followed by the user, and a list of other users who follow this user. The microblog information data includes the ID of the microblog, the user account of the post, the ID of the forwarded microblog, the content of the microblog, the time of posting, the place of posting, the device used to post the microblog, and the user account of @. There are a large number of relationships between data in this data set: the following relationship between users, the publishing relationship between users and Weibo, the reposting re...