A method and system for storing and analyzing measurement data format mdf files

A data file and format file technology, applied in the input/output process of data processing, database indexing, structured data retrieval, etc., can solve the problems of increased storage space, poor data reliability, and inability to read data, and reduce the occupation. The effect of space, reducing cache invalidation, and convenient storage expansion

Active Publication Date: 2021-04-30
昆易电子科技(上海)有限公司
View PDF8 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] (2) Storage expansion and backup are extremely complicated;
[0005] (3) Too much dependence on the database, unable to read data without connecting to the database;
[0006] (4) Difficulty in moving data;
[0007] (5) The storage space occupied by the data is larger than that of the original data, and the efficiency of analysis and query is limited by the processing capacity of the database;
[0008] (6) Expansion of database clusters, increased complexity of operation and maintenance, and poor data reliability; and
[0009] (7) In order to ensure the processing capacity of the database, small or large computers can be used, resulting in high system construction costs
[0010] In addition, since relational databases usually use row storage, all data needs to be scanned during query, which is inefficient

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A method and system for storing and analyzing measurement data format mdf files
  • A method and system for storing and analyzing measurement data format mdf files
  • A method and system for storing and analyzing measurement data format mdf files

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0056] The present invention will be described in detail below in conjunction with the accompanying drawings, and the features of the present invention will be further revealed in the following detailed description.

[0057] figure 1 is an example architecture diagram of a system 100 for storing and analyzing MDF files according to an embodiment of the present invention. Such as figure 1 As shown in , the system 100 of the present invention includes: a data collector 101 , a data processing platform 102 , a data parser 103 , a database 104 , a distributed file storage system 105 and a distributed file computing system 106 .

[0058]The data collector 101 is used to collect MDF data files and upload the collected data files to the data processing platform, wherein the collected MDF data files are disorderly and chaotic. Here, Measurement Data Format (MDF) is a dedicated standard format defined by ASAM (Association for Automation and Measurement Systems) for recording and tran...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a method and system for storing and analyzing measurement data format MDF files. The method includes: collecting and uploading MDF data files; sorting the collected MDF data files; extracting metadata from the sorted MDF data files and storing the extracted metadata; Convert the electrical signal value of each record in the file into a physical value; generate a self-parsed columnar storage format file and store the generated columnar storage format file in a distributed storage system; and use a distributed computing system to analyze all Stored columnar storage format files for calculation.

Description

technical field [0001] The present invention relates to the field of data storage and analysis, more particularly, to a method and system for storing and analyzing measurement data format MDF files. Background technique [0002] MDF file is a file format used to record data. Its main feature is that it takes up less space with the same amount of data. At present, the existing storage and analysis scheme is to analyze and store MDF files in a distributed database (for example, MySQL), and then query and analyze through SQL statements. However, this solution has the following problems: [0003] (1) It is necessary to know the data structure of the data to be stored in advance; [0004] (2) Storage expansion and backup are extremely complicated; [0005] (3) Too much dependence on the database, unable to read data without connecting to the database; [0006] (4) Difficulty in moving data; [0007] (5) The storage space occupied by the data is larger than that of the origin...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F3/06G06F16/22G06F16/27
CPCG06F3/0604G06F3/061G06F3/067G06F16/2282G06F16/27
Inventor 胡庆刚李宗阳李康
Owner 昆易电子科技(上海)有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap