Method, device and equipment for analyzing measurement data format (MDF) file and medium

Abstract

The application discloses an analysis method, device and equipment of a measurement data format (MDF) file and a medium. The method comprises the following steps: parsing each MDF file imported by a test system, generating a standard data table and metadata corresponding to each MDF file according to the parsing result, and storing the metadata into a metadata database; generating a column storage file corresponding to each standard data table, and storing each column storage file into a query database; determining a target analysis component according to an analysis requirement selected by a user, and determining a plurality of target column storage files in the query database according to the analysis requirement and the metadata database; and analyzing each target column storage file by using the target analysis component to obtain an analysis result. By using the technical scheme, the MDF file can be converted into an open source format for storage, so that the data in the MDF file can be parsed by using a simple data analysis tool, and the query efficiency of the data in the MDF file is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to an analysis method and device for a measurement data format (MDF) file, equipment and a medium. BACKGROUND

[0002] In the process of automobile testing, MDF (Measurement Data Format) files are often used to store internal measurement data of a controller.

[0003] However, the storage format of the MDF file is a private protocol format, and the query and analysis of the MDF file need to be performed with the aid of special commercial software. If a large number of MDF files need to be queried under specific conditions, a more expensive commercial statistical analysis system needs to be purchased, or the files can only be opened and queried one by one manually. SUMMARY

[0004] The present application provides an analysis method and device for a measurement data format (MDF) file, equipment and a medium, which can convert the MDF file into an open source format for storage, so that the data in the MDF file can be parsed by a simple data analysis tool, and the query efficiency of the data in the MDF file can be effectively improved.

[0005] According to an aspect of the present application, an analysis method for an MDF file is provided, which comprises:

[0006] parsing each MDF file imported by a test system, generating a standard data table and metadata corresponding to each MDF file according to the parsing result, and storing the metadata into a metadata database;

[0007] generating a column storage file corresponding to each standard data table, and storing each column storage file into a query database;

[0008] determining a target analysis component according to an analysis requirement selected by a user, and determining a plurality of target column storage files in the query database according to the analysis requirement and the metadata database;

[0009] analyzing each target column storage file by using the target analysis component to obtain an analysis result.

[0010] According to another aspect of the present application, an analysis device for an MDF file is provided, which comprises:

[0011] a file parsing module, configured to parse each MDF file imported by a test system, generate a standard data table and metadata corresponding to each MDF file according to a parsing result, and store the metadata into a metadata database;

[0012] A column storage file generation module is configured to generate column storage files corresponding to the standard data tables and store the column storage files in the query database.

[0013] A column storage file selection module is configured to determine a target analysis component according to the analysis requirement selected by the user, and determine a plurality of target column storage files in the query database according to the analysis requirement and the metadata database.

[0014] An analysis result acquisition module is configured to analyze the target column storage files by using the target analysis component, and acquire an analysis result.

[0015] According to another aspect of the present application, an electronic device is provided, which comprises:

[0016] at least one processor; and

[0017] a memory connected to the at least one processor in communication; wherein

[0018] the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the analysis method of the MDF file according to any one of the embodiments of the present application.

[0019] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to execute the analysis method of the MDF file according to any one of the embodiments of the present application when executed by the processor.

[0020] The technical solution of the embodiments of the present application can convert the MDF file into an open source format file by analyzing the MDF file into standard data tables and metadata, storing the metadata into the metadata database, generating column storage files according to the standard data tables and storing the column storage files into the query database, determining a target analysis component and target column storage files according to the analysis requirement of the user, and acquiring an analysis result, thereby solving the problem that the MDF file cannot be analyzed by using a general analysis software in the prior art, enabling efficient analysis and query of the data in the MDF file, solving the problem that MDF files of different channels cannot be stored in the same database, and effectively improving the query efficiency of the data in the MDF file.

[0021] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments description. Obviously, the drawings in the following description only show some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative effort based on these drawings.

[0023] Figure 1 is a flow chart of an MDF file analysis method according to an embodiment of the present application;

[0024] Figure 2 is a flow chart of another MDF file analysis method according to another embodiment of the present application;

[0025] Figure 3 is a structural schematic diagram of an MDF file analysis device according to an embodiment of the present application;

[0026] Figure 4 is a structural schematic diagram of an electronic device implementing the MDF file analysis method according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the person skilled in the art better understand the present application, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should be within the scope of the present application.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily mean a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0029] Embodiment one

[0030] Figure 1A flowchart of an MDF file analysis method provided for Embodiment One of the present application. This embodiment can be applied to the case where data in an MDF file is stored in an open source format to facilitate analysis of the MDF file. The method can be executed by an MDF file analysis device, which can be implemented in the form of hardware and / or software and can be configured in a computer or processor with data processing capabilities. As shown in FIG. 10, the method comprises: Figure 1

[0031] S110, parsing each MDF file imported by the test system, generating a standard data table and metadata corresponding to each MDF file according to the parsing result, and storing the metadata in a metadata database.

[0032] Optionally, the MDF file can be used to store test data of each test, such as vehicle model, vehicle speed, test parts, test conditions, test time, test location, vehicle temperature, data acquisition period, and various types of test data.

[0033] It can be understood that, in order to solve the problem that the MDF file needs to be analyzed by commercial software, the MDF file can be converted into an open source format file for storage, so that the file can be quickly read during file analysis.

[0034] Optionally, the MDF file imported by the test system at a time can be a single file or multiple files. In the embodiments described in the present application, each MDF file can be converted into a corresponding open source format file, and the present application selects an open source format file supporting columnar storage, such as a parquet format file, to facilitate subsequent rapid data analysis.

[0035] Optionally, the vehicle model, test time, test location, test conditions, and test parts in the MDF file, which have certain identification capabilities, can be used as metadata of the MDF file.

[0036] Optionally, the metadata of each MDF file can be stored in a metadata database.

[0037] ​Optionally, the test data in the MDF file can be generally stored in the form of a table, each column of the table can store multiple test data of a test item, and the test item can include but is not limited to vehicle speed, vehicle temperature, etc. However, due to different data collection periods of different test items in the vehicle test process, the amount of data collected by different test items can be different. For example, in a 1-minute test period, the vehicle speed collection period is 10 ms, and the temperature collection period is 100 ms. Therefore, the vehicle speed item in the table will store 100 pieces of data, but the temperature will store 10 pieces of data. In order to facilitate subsequent column storage, the irregular table needs to be regularized to a standard data table, that is, the last data in each column of the table is aligned. In the standard data table, the amount of data under each test item is the same. In the foregoing example, a vehicle speed collection period of 10 ms, a temperature collection period of 100 ms, or another time period, such as 50 ms, can be selected as a reference to expand or reduce the data to achieve data alignment. When the vehicle speed collection period of 10 ms is selected as the reference, the amount of temperature data is 1 / 10 of the amount of vehicle speed data, because the temperature data is collected once every 100 ms. Therefore, the temperature data can be expanded. The expansion method can include fitting a curve, repeatedly filling the last collected data, calculating an average value, etc., but is not limited to the above methods.

[0038] S120, generating a column storage file corresponding to each standard data table, and storing each column storage file in a query database.

[0039] Optionally, the column storage file is an open source storage file, and the column storage form is selected to facilitate subsequent data searching and analysis.

[0040] Optionally, the standard data table is composed of multiple data columns, and the column storage file can include multiple column storage spaces. Each column of data in the standard data table can be stored in a storage space in the column storage file.

[0041] The advantage of such a setting is that in a traditional data table, if the data of a test item is to be read, the data of each row needs to be read, and the data of the test item needs to be extracted from each row of data and combined. However, the column storage method used in the present application can extract the data in the storage space matched with the test item in the column storage file, which greatly saves computing resources and improves data analysis efficiency.

[0042] It can be understood that one MDF file can be stored as one column storage file, and each column storage file can be stored in one query database, effectively solving the problem that MDF files with different channels cannot be stored in the same database in the prior art.

[0043] S130, determine a target analysis component according to the analysis requirement selected by the user, and determine a plurality of target column storage files in the query database according to the analysis requirement and the metadata database.

[0044] It can be understood that if the data amount of the column storage file is small, the data in the query database can be stored locally on the computer, and if the data amount is large, it can be stored in the server. The existing analysis components, such as Python or R script query mode, and Arrow and duckdb components are suitable for analysis and query of locally stored files, but if the data amount is large and cross-server analysis and query is required, impala, clickhouse and other big data components need to be selected for query.

[0045] Further, the analysis component can also be determined according to the query requirement of the user. For example, for part of the query requirement, if the SQL (Structured Query Language) statement query is more accurate and convenient, the analysis component supporting the SQL statement query can be selected as the target component.

[0046] Optionally, the file identifier of the MDF file generally includes data such as date, vehicle type and experimental purpose. After obtaining the query requirement of the user, the target MDF file can be determined by preliminary matching according to the file identifier of the MDF file.

[0047] Optionally, the mapping relationship between the MDF file and the column storage file can be stored through metadata. The metadata can store the file identifier of the MDF file and the file identifier of the column storage file. When the user needs to query the target MDF file, the target metadata can be located in the metadata database through the file identifier of the MDF file, and the target column storage file in the query database can be determined through the file identifier of the target metadata.

[0048] S140, analyze each target column storage file by using the target analysis component to obtain an analysis result.

[0049] Optionally, the column storage file generally further includes an index, and the index records the data range of each test item in the column storage file. For example, in the current column storage file, the speed record value is in the range of 90km / h-120km / h. When the column storage file is analyzed, the analysis requirement of the user and the index can be used to preliminarily judge whether the column storage file meets the analysis requirement. If the user wants to analyze the speed in the range of 60km / h-80km / h, the column storage file can be directly abandoned. If the user wants to analyze the speed in the range of 90km / h-100km / h, the column storage file can be further analyzed.

[0050] The advantage of the arrangement is that the target column storage file can be quickly determined to meet the analysis requirement by arranging the index in the column storage file, and the analysis of invalid column storage files is avoided, and the analysis efficiency is improved.

[0051] The technical scheme of the embodiment of the application can convert the MDF file into an open source format file by analyzing the MDF file into a standard data table and metadata, storing the metadata into a metadata database, generating a column storage file according to the standard data table and storing the column storage file into a query database, determining a target analysis component and a target column storage file according to the analysis requirement of a user, and obtaining an analysis result, thereby solving the problem that the MDF file cannot be analyzed by using a general analysis software in the prior art, and achieving efficient analysis and query of the data in the MDF file, solving the problem that MDF files of different channels cannot be stored in the same database, and effectively improving the query efficiency of the data in the MDF file.

[0052] Embodiment Two

[0053] Figure 2 A flowchart of an analysis method of an MDF file provided by the embodiment two of the application is shown in the figure, and the embodiment specifically describes a method of converting the MDF file into a column storage file based on the above-mentioned embodiment. As shown in the figure, the method comprises: Figure 2

[0054] S210, each MDF file imported by the test system is analyzed, a standard data table and metadata corresponding to each MDF file are generated according to the analysis result, and the metadata is stored into a metadata database.

[0055] The analysis of each MDF file imported by the test system, the generation of the standard data table and the metadata corresponding to each MDF file according to the analysis result, and the storage of the metadata into the metadata database can comprise:

[0056] Whenever the target MDF file imported by the test system is detected, if it is determined that the target MDF file is a convertible file, the sampling period of each data column of the target MDF file is determined;

[0057] The target time period is determined according to the sampling period and a preset period selection rule, and the data in each data column is aligned according to the target time period;

[0058] Each data column after the alignment is combined to generate a standard data table corresponding to the target MDF file;

[0059] A plurality of metadata items are determined in the target MDF file according to a preset metadata field;

[0060] ​Extract identification information of the target MDF file, combine the identification information with each metadata item to form metadata matched with the target MDF file, and store the metadata into the metadata database.

[0061] Optionally, the MDF file can become an invalid file or an empty packet file due to data transmission errors, and such files cannot be converted and can be regarded as abnormal files.

[0062] Optionally, the metadata field can be understood as a field capable of identifying each metadata, for example, vehicle model, test time, test location, etc., and the metadata item is the value of the metadata field, for example, H5 (vehicle model), 20230101 (test time), and test No. 1 hall (test location).

[0063] The MDF files imported by the test system are parsed, and the standard data table corresponding to each MDF file and the metadata are generated according to the parsing result, and the metadata is stored into the metadata database, and the method can further include:

[0064] Whenever the target MDF file imported by the test system is detected, if it is determined that the target MDF file is an abnormal file, the abnormal information of the target MDF file is obtained.

[0065] The identification information in the target MDF file is combined with the abnormal information to form abnormal metadata, and the abnormal metadata is stored into the metadata database.

[0066] S220, obtaining the column field name of each data column in the standard data table matched with the target MDF file and the data item of each data column.

[0067] S230, determining the column storage position matched with each column field name in the initial storage file, and storing the data item matched with each column field name in the corresponding column storage position to generate a column storage file.

[0068] S240, generating a file identification uniquely matched with the column storage file, and adding the file identification to the column storage file.

[0069] S250, generating an index of the column storage file according to the data item stored in each storage position and the column field name matched with each storage position, and adding the index to the column storage file.

[0070] S260, storing the column storage file containing the file identification and the index into the query database.

[0071] S270, establishing a unique mapping relationship between each metadata and each column storage file according to the identification information of each MDF file and the file identification of the column storage file matched with each MDF file.

[0072] S280, determine the analysis difficulty and the data requirement according to the analysis requirement selected by the user, and determine the target analysis component among the plurality of analysis components according to the analysis difficulty and the data requirement.

[0073] S290, determine the analysis keyword according to the analysis requirement selected by the user, and acquire a plurality of target metadata matching the analysis keyword in the metadata database.

[0074] S2100, determine a plurality of target column storage files in the query database according to the mapping relationship between the metadata and the column storage files.

[0075] S2110, acquire the data range of the target analysis data, and determine whether there is a target analysis data value in the target column storage file within the data range according to the index of the target column storage file; if yes, execute S2120, and if no, execute S2130.

[0076] S2120, acquire the analysis result by using the target analysis component until the analysis result of all the target column storage files is acquired.

[0077] S2130, skip the target column storage file, analyze the next target column storage file until the analysis result of all the target column storage files is acquired.

[0078] The technical scheme of the embodiment of the application can convert the MDF file into an open source format file by analyzing the MDF file into a standard data table and metadata, storing the metadata into a metadata database, generating a column storage file according to the standard data table and storing the column storage file into a query database, determining the target analysis component and the target column storage file according to the analysis requirement of the user, and acquiring the analysis result, thereby solving the problem that the MDF file cannot be analyzed by using a general analysis software in the prior art, realizing efficient analysis and query of the data in the MDF file, solving the problem that MDF files of different channels cannot be stored in the same database, and effectively improving the query efficiency of the data in the MDF file.

[0079] Embodiment three

[0080] Figure 3 A structural schematic diagram of an MDF file analysis device provided by the embodiment three of the application is shown in FIG. 3. Figure 3 As shown in the figure, the device comprises a file analysis module 310, a column storage file generation module 320, a column storage file selection module 330, and an analysis result acquisition module 340.

[0081] The file analysis module 310 is configured to analyze each MDF file imported by the test system, generate a standard data table and metadata corresponding to each MDF file according to the analysis result, and store the metadata into the metadata database.

[0082] The column storage file generation module 320 is configured to generate a column storage file corresponding to each standard data table, and store each column storage file into the query database.

[0083] The column storage file selection module 330 is configured to determine a target analysis component according to the analysis requirement selected by the user, and determine a plurality of target column storage files in the query database according to the analysis requirement and the metadata database.

[0084] The analysis result acquisition module 340 is configured to analyze each target column storage file by using the target analysis component, and acquire an analysis result.

[0085] The technical scheme of the embodiment of the present application can convert the MDF file into an open source format file by analyzing the MDF file into a standard data table and metadata, storing the metadata into the metadata database, generating a column storage file according to the standard data table and storing the column storage file into the query database, determining a target analysis component and a target column storage file according to the analysis requirement of the user, and acquiring an analysis result, thereby solving the problem that the MDF file cannot be analyzed by using a general analysis software in the prior art, realizing efficient analysis and query of the data in the MDF file, solving the problem that MDF files of different channels cannot be stored in the same database, and effectively improving the query efficiency of the data in the MDF file.

[0086] On the basis of the above-mentioned embodiments, the file analysis module 310 can be specifically configured to:

[0087] Whenever it is detected that the test system imports a target MDF file, if it is determined that the target MDF file is a convertible file, the sampling periods of each data column of the target MDF file are determined;

[0088] The target time period is determined according to each sampling period and a preset period selection rule, and the data in each data column is aligned according to the target time period;

[0089] Each data column after the alignment is combined to generate a standard data table corresponding to the target MDF file;

[0090] A plurality of metadata items are determined in the target MDF file according to a preset metadata field;

[0091] Extract identification information of the target MDF file, combine the identification information with each metadata item, construct metadata matched with the target MDF file, and store the metadata into the metadata database.

[0092] On the basis of the above embodiments, the column storage file generation module 320 can be specifically used for:

[0093] Obtaining column field names of each data column in the standard data table matched with the target MDF file and data items of each data column;

[0094] Determining column storage positions matched with each column field name in the initial storage file, storing data items matched with each column field name in the corresponding column storage positions, and generating a column storage file;

[0095] Generating a file identification uniquely matched with the column storage file, and adding the file identification to the column storage file;

[0096] Generating an index of the column storage file according to the data items stored in each storage position and the column field names matched with each storage position, and adding the index to the column storage file;

[0097] Storing the column storage file containing the file identification and the index into the query database.

[0098] On the basis of the above embodiments, the mapping relationship establishment module can be further included, and after generating the column storage file according to the standard data table and storing the column storage file into the query database, the mapping relationship establishment module can be specifically used for:

[0099] Establishing a unique mapping relationship between each metadata and each column storage file according to the identification information of each MDF file and the file identification of the column storage file matched with each MDF file.

[0100] On the basis of the above embodiments, the column storage file selection module 330 can be specifically used for:

[0101] Determining an analysis difficulty and a data requirement according to the analysis requirement selected by the user, and determining a target analysis component in a plurality of analysis components according to the analysis difficulty and the data requirement;

[0102] Determining an analysis keyword according to the analysis requirement selected by the user, and obtaining a plurality of target metadata matched with the analysis keyword in the metadata database;

[0103] Determining a plurality of target column storage files in the query database according to the mapping relationship between the metadata and the column storage file.

[0104] On the basis of the above embodiments, the analysis result acquisition module 340 can be specifically used for:

[0105] acquire a data range of the target analysis data, and determine whether the target analysis data value located in the data range exists in the target column storage file according to an index of the target column storage file;

[0106] If yes, acquire the analysis result by using the target analysis component until the analysis result of all the target column storage files is acquired; if no, skip the target column storage file, and analyze the next target column storage file until the analysis result of all the target column storage files is acquired.

[0107] On the basis of the above-mentioned embodiments, the file parsing module 310 can also be used for:

[0108] Whenever it is detected that the test system imports the target MDF file, if it is determined that the target MDF file is an abnormal file, acquire abnormal information of the target MDF file;

[0109] Combine the identification information in the target MDF file with the abnormal information to form abnormal metadata, and store the abnormal metadata into the metadata database.

[0110] The MDF file analysis device provided in the embodiments of the present application can execute the MDF file analysis method provided in any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0111] Embodiment Four

[0112] Figure 4 A structural schematic diagram of an electronic device 10 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present application described and / or claimed in this document.

[0113] As Figure 4As shown, the electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor 11, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0114] Various components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, a speaker, etc., a storage unit 18, such as a magnetic disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0115] The processor 11 can be various general and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the analysis method of the MDF file as described in embodiments of the present application.

[0116] In some embodiments, the analysis method of the MDF file can be implemented as a computer program that is tangibly embodied in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the analysis method of the MDF file described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the analysis method of the MDF file by any other appropriate means, such as by means of firmware. That is:

[0117] The MDF files imported by the test system are parsed, and standard data tables and metadata corresponding to each MDF file are generated according to the parsing results, and the metadata is stored in the metadata database;

[0118] Column storage files corresponding to each standard data table are generated, and each column storage file is stored in the query database;

[0119] According to the analysis requirement selected by the user, a target analysis component is determined, and a plurality of target column storage files in the query database are determined according to the analysis requirement and the metadata database;

[0120] The target analysis component is used to analyze each target column storage file, and an analysis result is obtained.

[0121] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuitry, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0122] Computer programs used to implement the processes of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed, implements the functions / acts specified in the flow diagrams and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as part of a standalone software package, or entirely on a remote machine or server.

[0123] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0124] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0125] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0126] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0127] It should be understood that the various forms of flow shown above can be used to reorder, add or delete steps. For example, each step described in the present application can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which is not limited herein.

[0128] The above detailed description does not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. An analysis method of a measurement data format (MDF) file, characterized by, The method comprises the following steps: parsing each MDF file imported by the test system, generating a standard data table and metadata corresponding to each MDF file according to the parsing result, and storing the metadata into a metadata database; generating a column storage file corresponding to each standard data table and storing each column storage file into a query database; determining a target analysis component according to the analysis requirement selected by a user, and determining a plurality of target column storage files in the query database according to the analysis requirement and the metadata database; analyzing each target column storage file by using the target analysis component to obtain an analysis result; wherein the parsing of each MDF file imported by the test system, the generation of a standard data table and metadata corresponding to each MDF file according to the parsing result, and the storage of the metadata into a metadata database comprise the following steps: when it is detected that the test system imports a target MDF file, if it is determined that the target MDF file is a convertible file, determining a sampling period of each data column of the target MDF file; determining a target time period according to each sampling period and a preset period selection rule, and aligning the data in each data column according to the target time period; combining each data column after the alignment to generate a standard data table corresponding to the target MDF file; determining a plurality of metadata items in the target MDF file according to a preset metadata field; extracting identification information of the target MDF file, combining the identification information with each metadata item to form metadata matched with the target MDF file, and storing the metadata into the metadata database.

2. The method of claim 1, wherein, The generation of a column storage file corresponding to each standard data table and the storage of each column storage file into a query database comprise the following steps: obtaining a column field name of each data column in the standard data table matched with the target MDF file and a data item of each data column; determining a column storage position matched with each column field name in an initial storage file, storing the data item matched with each column field name in the corresponding column storage position to generate a column storage file; generating a file identification uniquely matched with the column storage file, and adding the file identification into the column storage file; generating an index of the column storage file according to the data item stored in each storage position and the column field name matched with each storage position, and adding the index into the column storage file; storing the column storage file containing the file identification and the index into the query database.

3. The method of claim 2, wherein, After the generation of a column storage file according to the standard data table and the storage of the column storage file into the query database, the method further comprises the following steps: establishing a unique mapping relationship between each metadata and each column storage file according to the identification information of each MDF file and the file identification of the column storage file matched with each MDF file.

4. The method of claim 3, wherein, The determination of a target analysis component according to the analysis requirement selected by a user and the determination of a plurality of target column storage files in the query database according to the analysis requirement and the metadata database comprise the following steps: determining an analysis difficulty and a data requirement according to the analysis requirement selected by a user, and determining a target analysis component in a plurality of analysis components according to the analysis difficulty and the data requirement; According to the analysis requirement selected by the user, the analysis keyword is determined, and a plurality of target metadata matching the analysis keyword is obtained from the metadata database; According to the mapping relationship between the metadata and the column storage file, a plurality of target column storage files in the query database is determined.

5. The method of claim 4, wherein, The target analysis component is used to analyze each target column storage file to obtain an analysis result, including: The data range of the target analysis data is obtained, and whether there is a target analysis data value in the target column storage file within the data range is determined according to the index of the target column storage file; If yes, the analysis result is obtained by using the target analysis component until the analysis result of all target column storage files is obtained; if no, the target column storage file is skipped, and the next target column storage file is analyzed until the analysis result of all target column storage files is obtained.

6. The method of claim 1, wherein, Each MDF file imported by the test system is parsed, and a standard data table corresponding to each MDF file and metadata are generated according to the parsing result, and the metadata is stored in the metadata database, and the method further includes: Whenever the target MDF file imported by the test system is detected, if it is determined that the target MDF file is an abnormal file, the abnormal information of the target MDF file is obtained; The identification information in the target MDF file is combined with the abnormal information to form abnormal metadata, and the abnormal metadata is stored in the metadata database.

7. An analysis apparatus of a measurement data format (MDF) file, characterized by, It includes: The file parsing module is used to parse each MDF file imported by the test system, generate a standard data table corresponding to each MDF file and metadata according to the parsing result, and store the metadata in the metadata database; The column storage file generation module is used to generate column storage files corresponding to each standard data table, and store each column storage file in the query database; The column storage file selection module is used to determine the target analysis component according to the analysis requirement selected by the user, and determine a plurality of target column storage files in the query database according to the analysis requirement and the metadata database; The analysis result acquisition module is used to analyze each target column storage file by using the target analysis component to obtain an analysis result; The file parsing module is specifically used for: Whenever the target MDF file imported by the test system is detected, if it is determined that the target MDF file is a convertible file, the sampling period of each data column of the target MDF file is determined; According to each sampling period and a preset period selection rule, a target time period is determined, and the data in each data column is aligned according to the target time period; The aligned each data column is combined to generate a standard data table corresponding to the target MDF file; A plurality of metadata items in the target MDF file is determined according to a preset metadata field; The identification information of the target MDF file is extracted, the identification information is combined with each metadata item to form metadata matching the target MDF file, and the metadata is stored in the metadata database.

8. A computer device, comprising: The computer device includes: At least one processor; and The memory is connected in communication with the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the analysis method of the measurement data format MDF file in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to implement the analysis method of the measurement data format MDF file in any one of claims 1-6 when executed.

Images