A high magnitude of data management method

A massive data management and data technology, applied in the direction of electrical digital data processing, special data processing applications, instruments, etc., can solve the problems of slow insertion speed, data insertion failure, and low insertion efficiency, so as to increase the insertion speed and improve the insertion efficiency. The effect of speed and query speed improvement

Inactive Publication Date: 2007-12-12
上海全成通信技术有限公司
View PDF0 Cites 24 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In the existing technology, when the mobile short message CDR data file obtained from the gateway is directly inserted into the database, due to some invalid data in the CDR, such as invalid lines, invalid fields, etc., and there are more than 20 fields in the CDR, each field Any error will cause the failure of inserting data, and the application will throw an exception. The exception processing will take up more processing time of the computer, thus affecting the insertion speed of the database.
On the other hand, each time a CDR record is inserted, because there is a large overhead in each interaction with the database, the insertion efficiency is very low
When the mobile CDR inserts millions of pieces of data into an empty database table every day, the insertion speed is fast. As the date goes by, the data in the table will increase, and the insertion speed will become slower and slower, thus Leading to a backlog of data that cannot be processed in a timely manner

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A high magnitude of data management method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0015] The core of the present invention is to divide a large database table into multiple partitions, each of which is equivalent to a small database table, data insertion is performed by partition, and the partition is divided by date. If you need to save 90 days of data, create 91 partitions, and insert the data for each day into their respective partitions. When 90 days of data are inserted into the 90 partitions, the 91st day data is inserted into the 91st partition, and then the first one is deleted. Partitioned data, so that it can always save the last 90 days of data. The partition number is generated based on the date and the value is cycled between 1-91. In order to further improve the efficiency of database parallel insertion and query, we create five data table spaces, one partition and one table space, and the partition numbers separated by five use the same table space, that is, the sixth partition uses the same table as the first partition Space, other analogy.

[0...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

A large amount data management method divides the data base into several zones, with each being a small data base marked with dates, data inserted into the zones in sequence and updated in the number limit, and establishing five more data base list space for further improving the data base parallel and inquiry efficiency. It can avoid the slowing down of overall application system efficiency due to large amount of data, with reduced running expenses and improved efficiency.

Description

Technical field [0001] The invention relates to a computer operating system, in particular to a mass data management method. Background technique [0002] Traditional large-scale application software systems manage and query and analyze the data collected by their application management objects. Generally speaking, the amount of management and query data is very huge. For example, import and query the CDR (Call Detail Record) data of the mobile SMS gateway. In the existing technology, when the mobile SMS CDR data file obtained from the gateway is directly inserted into the database, there are some invalid data in the CDR, such as invalid rows, invalid fields, etc., and there are more than 20 fields in the CDR, each field Errors will cause the insertion of data to fail, and the application will throw exceptions. The exception processing will take up more processing time on the computer, which will affect the insertion speed of the database. On the other hand, each time a CDR recor...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
Inventor 冯谧
Owner 上海全成通信技术有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap