Method and system for storing analytic measurement data format MDF file

A data file and format file technology, applied in the input/output process of data processing, database indexing, structured data retrieval, etc., can solve the problems of inability to read data, increase storage space, poor data reliability, etc., and achieve storage expansion. Convenience, reduce footprint, and reduce the effect of cache invalidation

Active Publication Date: 2021-02-02
昆易电子科技(上海)有限公司
View PDF8 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] (2) Storage expansion and backup are extremely complicated;
[0005] (3) Too much dependence on the database, unable to read data without connecting to the database;
[0006] (4) Difficulty in moving data;
[0007] (5) The storage space occupied by the data is larger than that of the original data, and the efficiency of analysis and query is limited by the processing capacity of the database;
[0008] (6) Expansion of database clusters, increased complexity of operation and maintenance, and poor data reliability; and
[0009] (7) In order to ensure the processing capacity of the database, small or large computers can be used, resulting in high system construction costs
[0010] In addition, since relational databases usually use row storage, all data needs to be scanned during query, which is inefficient

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for storing analytic measurement data format MDF file
  • Method and system for storing analytic measurement data format MDF file
  • Method and system for storing analytic measurement data format MDF file

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0056]The present invention will be described in detail below in conjunction with the accompanying drawings, and the features of the present invention will be further revealed in the following detailed description.

[0057]figure 1 It is an example architecture diagram of a system 100 for storing and analyzing MDF files according to an embodiment of the present invention. Such asfigure 1 As shown in the figure, the system 100 of the present invention includes: a data collector 101, a data processing platform 102, a data parser 103, a database 104, a distributed file storage system 105, and a distributed file computing system 106.

[0058]The data collector 101 is used to collect MDF data files and upload the collected data files to the data processing platform, where the collected MDF data files are disorderly. Here, the measurement data format (MDF) is a special standard format defined by ASAM (Automation and Measurement System Standards Association) for recording and transmitting measur...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a method and system for storing an analytical measurement data format MDF file. The method comprises the steps of collecting and uploading an MDF data file; sorting the collected MDF data files; extracting metadata from the sorted MDF data files and storing the extracted metadata; converting the electric signal value of each record in the sorted MDF data file into a physicalvalue; generating a self-parsed column type storage format file and storing the generated column type storage format file in a distributed storage system; and calculating the stored column type storage format file by utilizing a distributed calculation system.

Description

Technical field[0001]The present invention relates to the field of data storage and analysis, and more specifically, to a method and system for storing an MDF file of analysis measurement data format.Background technique[0002]MDF file is a file format used to record data. The main feature is that it takes up less space under the same amount of data. The existing storage and analysis solution is to parse the MDF file into a distributed database (for example, MySQL), and then query and analyze it through SQL statements. However, this solution has the following problems:[0003](1) The data structure of the data to be stored needs to be known in advance;[0004](2) Storage expansion and backup are extremely complicated;[0005](3) Too much dependence on the database, unable to read data without connecting to the database;[0006](4) Difficulties in data movement;[0007](5) The storage space occupied by data is larger than that of original data, and the efficiency of analysis and query is limite...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06G06F16/22G06F16/27
CPCG06F3/0604G06F3/061G06F3/067G06F16/2282G06F16/27
Inventor 胡庆刚李宗阳李康
Owner 昆易电子科技(上海)有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap