Quick analysis and storage method of big data based on database

A data analysis and big data technology, applied in the direction of electrical digital data processing, special data processing applications, instruments, etc., can solve the problems of single client processing that does not make full use of computer resources, unpredictable time delay, poor scalability and versatility, etc. Achieve the effects of configurable data structure, good scalability, and saving reading and writing time

Active Publication Date: 2015-05-13
VISION MICROSYST SHANGHAI
View PDF4 Cites 19 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] 1. Versatility: The files in the above patents need to set the data format, which has poor scalability and versatility;
[0005] 2. Processing efficiency: A large part of the design logic in the above patents is how to split and combine files, and the corresponding disk reading and writing and network transmission will consume a lot of time a

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Quick analysis and storage method of big data based on database

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0034] The present invention will be described in detail below in conjunction with specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0035] Such as figure 1 As shown, the database-based fast analysis and storage method for big data provided by the present invention specifically includes the following steps:

[0036] Step 1: According to the actual data content to be analyzed, define the data format into the file.

[0037] The data format is written in Extensible Markup Language in order to define data in any format. The data items involved in the data format must define their identification, name, type, length, whether there i...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

A quick analysis and storage method of big data based on a database includes the steps 1, according to data content actually to be analyzed, defining a data format to a file; 2, reading the data format from the file to a data structure constructed in a memory, as data analysis basis; 3, making preparation before data analysis; 4, analyzing data; 5, storing the data; 6, after a list in the step 5 finishes storage, clearing a results list, setting a processing completion mark to be false, setting a status of the processing completion mark to be idle, recovering the processing completion mark to a thread pool, and waiting for allocation of new data blocks. The method has the advantages that analytical speed is high, the analyzed data structure is configurable and highly universal, data storage redundancy is low, and stored results data are convenient to analyze afterwards.

Description

technical field [0001] The invention relates to a computer data processing technology, and more specifically, relates to solving the problems of fast analysis and post-event analysis of big data. Background technique [0002] In many projects, a large amount of data that needs to be processed at high speed will be encountered. If these data cannot be processed in time, it will have a fatal impact on the entire software system. The existing big data analysis solutions basically deal with fixed formats. Yes, the scalability compatibility is very poor, and the processing speed on a single computer is not fast enough, and the existing technology is not fully utilized to maximize the processing capacity of the computer. Distributed computing requires high network speed and stability, and An efficient and comprehensive data storage method is not provided, so the extraction of data after processing is not flexible. [0003] After searching, the application number is 200810097594.7...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/2282G06F16/2291
Inventor 彭成志刘钧钧咸峰
Owner VISION MICROSYST SHANGHAI
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap