A data processing and access method based on index cache

Abstract

The present application provides a data processing and access method based on index cache, which belongs to the field of avionics technology, and specifically includes splitting the data packets in the flight parameter data stream according to the channel number to form a number of channel data files; writing the parameter configuration file and the channel data file index, and writing the channel data file in the order of the channel number to form a final processing file; reading the final processing file during data playback, obtaining parameter information, reading the final processing file according to the final processing file position recorded in the parameter information, parsing the parameters according to the parameter configuration information in the parameter configuration file, and quickly reading the data in combination with the relative position and index of each channel during reading; generating a cache file, and the cache file is used to store the parsed data; when replaying the data again, the cache file is read first, and if the cache file does not exist, the step of reading the final processing file during data playback is entered. Through the processing scheme of the present application, the data processing speed is accelerated and the data access speed is improved.

Description

Technical Field

[0001] The present application relates to the field of avionics technology, and in particular to a multimedia stream data synchronous playback system and method based on data fusion. Background Art

[0002] Flight parameter data is collected and recorded by the flight parameter recording system, and plays an irreplaceable and important role in aviation accident investigation. In recent years, with the increase in the number of parameter collection, the sampling rate and the change in the type of data collection, the amount of airborne aviation data has increased, and the packet collection structure has also made the number of channels reach hundreds. The reading and writing of a large number of small files has a greater impact on the data processing speed. In order to quickly process and access large-capacity multi-channel data, it is necessary to improve the data processing method and access method to improve the data processing and access efficiency. Summary of the invention

[0003] In view of this, an embodiment of the present application provides a data processing and access method based on index cache, which provides a method for processing and organizing original data to generate an easily accessible data file format to facilitate rapid access to data during data playback.

[0004] The present application provides a data processing and access method based on index cache, the method comprising:

[0005] Step 1: split the data packets in the flight parameter data stream according to the channel number to form several channel data files;

[0006] Step 2: write the parameter configuration file and the channel data file index, and write the channel data file in the order of the channel numbers to form the final processing file;

[0007] Step 3: during data playback, the final processing file is read to obtain parameter information; according to the position of the final processing file recorded in the parameter information, the final processing file is read; parameters are parsed according to the parameter configuration information in the parameter configuration file; and data is quickly read in combination with the relative position and index of each channel during reading;

[0008] Step 4: Generate a cache file, which is used to store the parsed data;

[0009] Step 5: When replaying the data again, first read the cache file. If the cache file does not exist, go to step 3.

[0010] According to a specific implementation method of an embodiment of the present application, when the data packet is split, each packet of data only contains the total length of the current data packet, the relative timestamp and the original code data, and is saved in the channel data file corresponding to the corresponding data packet.

[0011] According to a specific implementation method of an embodiment of the present application, generating a final processing file is specifically as follows: writing in order the file index values ​​of each part of the final processing file, each parameter configuration file, the channel data file time index, and the index position of each data packet in the file, and finally writing the channel data file in order of the channel number.

[0012] According to a specific implementation method of an embodiment of the present application, the file index values ​​of each part in the final processing file include a parameter configuration file position, a channel data file time index position and a data start position.

[0013] According to a specific implementation method of an embodiment of the present application, the cache file is stored in a folder with the same directory and name as the parsed file, and is generated with the name of history name + channel number + parameter index number, and the history name is the name of the final processed file.

[0014] According to a specific implementation method of the embodiment of the present application, in the step five, if the cache file does not exist, the process proceeds to the step three, and the cache file is first deleted when the data is preprocessed again.

[0015] According to a specific implementation method of an embodiment of the present application, the parameter configuration file includes: parameter name, system to which the parameter belongs, channel to which the parameter belongs, parameter position and parameter parsing formula information.

[0016] Beneficial Effects

[0017] The data processing and access method based on index cache in the embodiment of the present application is designed for multi-channel data and provides powerful technical support for data playback. The main beneficial effects are as follows:

[0018] 1) Speed ​​up data processing

[0019] The present invention performs sequential reading and writing of multi-channel data, which can effectively improve processing efficiency.

[0020] 2) Improve data access speed

[0021] The data structure formed by the present invention can improve the speed of repeatedly viewing data.

[0022] 3) Improve user experience

[0023] By adopting the method of the present invention, the processing and playback speed of multi-channel data can be significantly improved, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 The figure is a flow chart of a data processing and access method based on index cache according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0027] The following describes the implementation methods of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, in the absence of conflict, the following embodiments and the features in the embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work belong to the scope of protection of the present application.

[0028] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present application, it should be understood by those skilled in the art that an aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein can be used to implement this device and / or practice this method.

[0029] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application. The drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0030] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0031] The present application embodiment provides a data processing and access method based on index cache, which provides a method for processing and arranging raw data to generate a data file format that is easy to access, so as to facilitate fast access to data during data playback. For specific method steps, refer to Figure 1 .

[0032] The method mainly includes the following steps:

[0033] Step 1: Split the data packets in the flight parameter data stream according to the channel number to form several channel data files. That is, split the original flight parameter data according to the data packet type. Each data packet corresponding to each channel has a channel data file. When split, each data packet only contains the total length of the current data packet, the relative timestamp, and the original code data. The channel data file index can be recorded every ten minutes.

[0034] Specifically, the data packets are classified according to the channel number label in the data packet header in the data stream, and the data of the data packets of different channels are arranged in chronological order (i.e., the recorded data stream order). The packet structure retains the total length, relative timestamp, and original data, and is saved in a temporary data file corresponding to the corresponding data packet, thereby realizing the splitting of the original data into several channel data files according to the data type package.

[0035] Step 2: Write the parameter configuration file and the channel data file index, and write the channel data file in the order of the channel numbers to form the final processing file.

[0036] Specifically, the final processing file is generated by writing the file index values ​​of each part, each parameter configuration file, the channel data file time index, and the index position of each data packet in the file in order, and finally writing the channel data file generated in the first step in the configuration order. Among them, the file index values ​​of each part in the final processing file include the parameter configuration file position, the channel data file time index position, and the data start position; the parameter configuration file mainly includes the parameter name, the system to which the parameter belongs, the channel to which the parameter belongs, the parameter position, and the parameter parsing formula information. Of course, this part of the content can also be adjusted according to actual needs.

[0037] Step 3: Read the final processing file during data playback to obtain parameter information. Read the final processing file according to the final processing file position recorded in the parameter information, parse the parameters according to the parameter configuration information in the parameter configuration file, and quickly read the data in combination with the relative position and index of each channel during reading. Specifically, obtain the parameter information by reading the parameter configuration part, obtain the corresponding channel information in the parameter information, determine the position of the final processing file where the parameter corresponding channel is located by reading the data start position and the channel index position, accurately obtain the channel data position within the specified time through the channel time index, obtain the parameter source code value through the parameter position information in the parameter information, and parse the parameter through the parameter parsing formula information.

[0038] Specifically, the parameter configuration and index information are read in the order of writing the file in step 2. The parameters selected by the user are classified, and the data belonging to the same channel belongs to one category. The original code data is read and parsed according to the category order (that is, the channel order) and the index information. When performing random reading, the query and reading are performed according to the index information. In this step, sequential reading and writing of multi-channel data can effectively improve processing efficiency.

[0039] Step 4: Generate a cache file, which is used to store the parsed data.

[0040] Specifically, a cache file is generated after data reading and parsing is completed. The cache file is stored in a folder with the same name as the parsed file. It is generated with the name of the resume name + channel number + parameter index number, and the physical quantity after parsing is written. The resume name is the final processed file name.

[0041] Step 5: When replaying the data again, first read the cache file, if the cache file does not exist, go to step 3. When replaying the data, the replayed data is recorded in the form of a cache file, which can reduce the time loss caused by multiple reading and writing.

[0042] When replaying the data again, it is necessary to check whether there is a cached file. If there is a cached file, read the cached file. If not, go to step 3 to read it. When re-preprocessing the data, first delete the cached file to avoid affecting the reading.

[0043] The present invention splits the original flight parameter data according to the data packet type and records the time index information of the channel data file. The configuration information, index information and channel data file are written into the file to avoid the slow processing speed caused by reading and writing a large number of small files. When the data is played back, the data that has been played back is recorded in the form of a cache file to reduce the time loss caused by multiple reading and writing.

[0044] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A data processing and access method based on index cache, characterized in that: The method comprises: Step 1: split the data packets in the flight parameter data stream according to the channel number to form several channel data files; Step 2: write the parameter configuration file and the channel data file index, and write the channel data file in the order of the channel number to form a final processing file; generating the final processing file specifically includes: writing the file index values ​​of each part of the final processing file, each parameter configuration file, the channel data file time index, and the index position of each data packet in the file in order, and finally writing the channel data file in the order of the channel number; Step 3: during data playback, the final processing file is read to obtain parameter information, the final processing file is read according to the position of the final processing file recorded in the parameter information, parameters are parsed according to the parameter configuration information in the parameter configuration file, and data is quickly read in combination with the relative position and index of each channel during reading; including: reading parameter configuration and index information in the order of writing the file in step 2, classifying the parameters selected by the user, data belonging to the same channel belongs to one category, reading and parsing the original code data according to the category order and index information, and querying and reading according to the index information during random reading; Step 4: Generate a cache file, which is used to store the parsed data; Step 5: When replaying the data again, first read the cache file. If the cache file does not exist, go to step 3.

2. The data processing and access method based on index cache according to claim 1, characterized in that: When the data packets are split, each packet of data only contains the total length of the current data packet, the relative timestamp and the original code data, and is saved in the channel data file corresponding to the corresponding data packet.

3. The data processing and access method based on index cache according to claim 1, characterized in that: The file index values ​​of each part in the final processing file include the parameter configuration file position, the channel data file time index position and the data start position.

4. The data processing and access method based on index cache according to claim 1, characterized in that: The cache file is stored in a folder with the same name as the parsed file, and is generated with the name of history name + channel number + parameter index number, where the history name is the name of the final processed file.

5. The data processing and access method based on index cache according to claim 1, characterized in that: In step 5, if the cache file does not exist, the process proceeds to step 3, and the cache file is first deleted when the data is preprocessed again.

6. The data processing and access method based on index cache according to any one of claims 1 to 5, characterized in that: The parameter configuration file includes: parameter name, parameter system, parameter channel, parameter location and parameter parsing formula information.

Images