Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Partitioning/sharding persistent scheme for high QPS (Query Per Second) data

A persistent, data technology, applied in the direction of digital data processing, structured data retrieval, special data processing applications, etc.

Inactive Publication Date: 2017-01-04
华云科技有限公司
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The current solution is to use multiple servers for partition storage processing, including the time partition storage method. The above method can only alleviate the problem of database storage speed drop in the initial stage, but cannot completely solve the problem. As the amount of data stored by the server group accumulates, the storage speed The drop problem will explode
[0005] In addition, data accumulation is inevitable in the system, and the problem of reduced storage speed can only be solved by separately processing the accumulated data and the current real-time new data.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Partitioning/sharding persistent scheme for high QPS (Query Per Second) data
  • Partitioning/sharding persistent scheme for high QPS (Query Per Second) data
  • Partitioning/sharding persistent scheme for high QPS (Query Per Second) data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0026] The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments, wherein the schematic embodiments and descriptions are only used to explain the present invention, but are not intended to limit the present invention.

[0027] Such as Figure 1-Figure 2 As shown, the partition fragmentation high QPS data persistence scheme described in this specific embodiment adopts the following method steps:

[0028] Step 1: The system data storage and processing are periodically segmented according to the time when the data is generated, and the data generated at each specific time can only be processed in the corresponding time slice database for business logic;

[0029] Step 2: The business logic processing process described in Step 1 is divided into three core stages:

[0030] The three core stages are the save stage, the stage of transferring to other persistent systems, and the stage of cleaning up the accumulated d...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to the technical field of software development, in particular to a partitioning / sharding persistent scheme for high QPS (Query Per Second) data. The partitioning / sharding persistent scheme is characterized by adopting the following method comprising the following steps: firstly, saving and processing system data, and periodically performing sharding processing on generation time of the data, wherein each datum generated in a specified time is only subjected to service logic processing in a corresponding time slice database; secondly, dividing the service logic processing process in the first step into three core stages: a saving stage, a stage of unloading into other persistent systems and a stage of cleaning up current time slice accumulating data. The partitioning / sharding persistent scheme has the advantages that a flow is simple; the problems except the problems related to data and time; the partitioning / sharding persistent scheme is irrelevant to other demands of the system; high QPS persistent data are kept into a database in the long-term operation process of the system; the working efficiency of the system is improved, and the like.

Description

[0001] 【Technical field】 [0002] The invention relates to the technical field of software development, in particular to a high-QPS data persistence scheme for partitioning and fragmentation. [0003] 【Background technique】 [0004] With the continuous development of social science and technology, database systems are widely used in various industries. When the database system saves persistent data, the saving speed (QPS) of the database will decrease geometrically as the number of records increases. The current solution is to use multiple servers for partition storage processing, including the time partition storage method. The above method can only alleviate the problem of database storage speed drop in the initial stage, but cannot completely solve the problem. As the amount of data stored by the server group accumulates, the storage speed The drop problem will pop up. [0005] In addition, data accumulation is inevitable in the system, and the problem of slowing down the ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F17/30
CPCG06F16/21
Inventor 柳志刚
Owner 华云科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com