Incremental processing method of hierarchical concept vectorization in personal big data management

Abstract

Provided is a personal big data management hierarchy concept vectorizing incrementation processing method. The method comprises the following steps that 1, when a system run for the first time, all concepts are vectorized, and all branching nodes are subjected to concept vector merging operation; 2, when a user operates a concept tree, the substeps of 2.1 obtaining concept vectors and total word number of operated nodes and father nodes thereof, 2.2 modifying the concept vectors of the father nodes according to a formula, 2.3 conducting recursive implementation from the substep 2.1 by taking the father nodes as the operated nodes till a root node and 2.4 updating an inverse document frequency vector are executed; 3, when errors are accumulated to a certain degree, the substeps of 3.1 obtaining current inverse document frequency vector and an inverse document frequency initial value vector, 3.2 updating all vector weights in a vector space in a batched mode and 3.3 updating the inverse document frequency initial value vector are executed. According to the method, the personal big data management hierarchy concept vectorizing incrementation calculation method is achieved, the concept vectors in the concept space can be rapidly adjusted, and the execution efficiency is improved.

Description

Technical field [0001] The invention relates to the management, organization, query and retrieval technology of personal big data, in particular to a hierarchical concept vectorization method based on a vector space model and an incremental calculation method thereof. Background technique [0002] With the development of information technology, personal data has exploded, including personal documents (text, images, voice), emails, health data, personal mobile phone contact information (WeChat, QQ), Internet data, etc., which have entered personal big data ( Personal big data) era; the development of wearable devices will further intensify the growth of data. People can record what they hear and see, and collect physiological health data throughout the day. How to manage and organize personal big data, through simple operations, can always get accurate, appropriate, complete, and high-quality information in the right place is a goal of the personal information management system. ...

AI Technical Summary

Problems solved by technology

[0005] Generally speaking, since the feature items of concepts are usually thousands, the calculation of the concept vectorization process is relatively large.

If the traditional document vectorization method is used for concept vectorization, when the number of concepts changes, such as the addition of new concepts or the deletion of old concepts, all existing concept vectors will be biased; if the vector space is rebuilt, the amount of calculation is usually bigger

[0006] In addition, traditional document vectorization techniques are mostly based on single-level document classification structures, which are not suitable for direct application to concept trees.

Images