Supercharge Your Innovation With Domain-Expert AI Agents!

Method for automatically fragmenting cold and hot data based on MongoDB

A technology of hot and cold data and data sharding, which is applied in the input/output process of data processing, electrical digital data processing, instruments, etc., can solve the problems of server load difference, unconsidered data access timeliness, etc., to avoid frequent access, Avoid the effect of data load imbalance

Pending Publication Date: 2019-12-20
上海新炬网络技术有限公司
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the Auto-Sharding mechanism only considers the balance of data volume and does not consider the timeliness of data access when data sharding and formulating data migration strategies, resulting in load differences between servers

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for automatically fragmenting cold and hot data based on MongoDB
  • Method for automatically fragmenting cold and hot data based on MongoDB
  • Method for automatically fragmenting cold and hot data based on MongoDB

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0024] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0025] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one of ordinary skill in the art that the present invention may be practiced without these specific details. Therefore, the specific details set forth are merely exemplary, and specific details may be varied in the spirit and scope of the invention and still be considered to be within the spirit and scope of the invention.

[0026] This embodiment provides a method for automatic sharding based on MongoDB cold and hot data, and establishes a new data migration strategy based on the heat load difference between data shards to avoid frequent access to hot tables and unbalanced data load caused by uneven data distribution. .

[0027] see now figure 1 , the technical scheme tha...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method for automatically fragmenting cold and hot data based on MongoDB. The method comprises the following steps: S101, scanning a hot data proportion value of each data fragment; S102, judging whether a hot data proportion value exceeds a preset threshold value or not, and ending if the hot data proportion value exceeds the preset threshold value; S103, determining a proper data fragment size according to the key interval of the data partition; and S104, migrating the maximum hot block data to the minimum cold block. According to the method for automatically fragmenting cold and hot data based on MongoDB, cold and hot data judgment is carried out on the access characteristics of the data through a naive Bayes algorithm. The proportion of the hot data in the datafragments is used as the thermal load value to determine the data migration opportunity, and a new data migration strategy is established according to the thermal load difference between the data fragments, so that frequent access of a thermal table and unbalanced data load caused by uneven data distribution are avoided.

Description

technical field [0001] The invention relates to a method for automatic sharding of cold and hot data, in particular to a method for automatic sharding of cold and hot data based on MongoDB. Background technique [0002] MongoDB is a database based on distributed file storage. MongoDB is a product between relational databases and non-relational databases. It is the most functional among non-relational databases and most similar to relational databases. The data structure it supports is very loose, which is a bson format similar to json, so it can store more complex data types. The biggest feature of MongoDB is that the query language it supports is very powerful. Its syntax is somewhat similar to object-oriented query language. It can almost realize most of the functions similar to single-table query of relational database, and it also supports indexing of data. [0003] MongoDB's Auto-Sharding mechanism can help MongoDB database solve the problem of high scalability and hi...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F3/06
CPCG06F3/0607G06F3/0644G06F3/067
Inventor 程永新孙玉颖阮飞鹏
Owner 上海新炬网络技术有限公司
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com