An aviation bus data processing method based on a time pyramid

Abstract

The present application relates to the field of data processing, specifically to an aviation bus data processing method based on time pyramid, which comprises the following steps: S1, cleaning and checking the data; S2, dividing the complex table structure according to columns; S3, constructing data combination of time column and data column; S4, constructing the bottom layer basic tile data of time pyramid data; S5, extracting the basic data to be processed in the form of frequency reduction by half each time to form multi-level frequency-decreasing extracted data, and the data forms multi-level LOD data; S6, processing the data in each level of LOD data according to the time block processing method of each level of tile construction; S7, serializing the tile data to the database to form hierarchical tile pyramid data. The present application realizes the tile segmentation of the aircraft bus data, and then processes the huge and complex bus data tiles, which can achieve the purpose of on-demand calling and on-demand taking in use.

Description

Technical Field

[0001] This invention relates to the field of data processing, and in particular to an aviation bus data processing method based on a time pyramid. Background Technology

[0002] Aircraft bus data is massive, typically a single flight lasting about two hours, resulting in decoded data ranging from tens to thousands of gigabytes. Different types of data are managed using two-dimensional tables, which are complex in their descriptions, often containing dozens to thousands of fields per file. Except for the time field, other columns describe a type of metric through one or more columns; for example, aircraft trajectory data includes a time column and x, y, and z coordinate columns. Traditionally, this type of data is used by extracting relevant data columns and time columns from the corresponding tables based on specific business needs, then applying the data to various metric analyses and visualizations. However, directly scheduling the raw data for graphical visualization of this large dataset is inefficient, consumes huge system resources, and is generally not feasible on ordinary graphics workstations, failing to meet the needs of on-demand scheduling and rapid display. Summary of the Invention

[0003] The purpose of this invention is to address the problems existing in the background technology by proposing an aviation bus data processing method based on a time pyramid, which realizes the tile segmentation of aircraft bus data, thereby processing the massive and complex bus data into tiles, so that it can be called on demand and retrieved as needed.

[0004] The technical solution of this invention, an aviation bus data processing method based on a time pyramid, includes the following steps:

[0005] S1. Clean and validate the data, removing invalid data;

[0006] S2. Divide the complex table structure into columns, with each column containing a type of bus information data;

[0007] S3. Construct a data combination with a time column and a data column, that is, attach the time column to each data column to form a data combination indexed by time;

[0008] S4. Take the combined time and data column data as the basic data to be processed, calculate the total duration of the data according to the start and end time intervals of the data, divide the data into n Block data segments according to the time, and then construct the bottom layer basic tile data of the time pyramid data.

[0009] S5. The basic data to be processed is thinned by halving the frequency each time to form multi-level thinned data with decreasing frequency. After each thinning and block division, the data forms multi-level LOD data.

[0010] S6. Divide the LOD data of each level into blocks according to time according to the method of constructing tiles for each level. Each data segment after being divided into blocks is called a data tile.

[0011] S7. Serialize the tile data into the database to form a hierarchical tile pyramid data.

[0012] Preferably, in S3, the time column can be combined with multiple related data columns to form a type of data.

[0013] Preferably, in S4, the number of data blocks can be defined by the user.

[0014] Preferably, in S7, the tile pyramid data serves as the basic data for visualization. During the visualization process, different levels of tile data can be called according to resolution requirements to achieve data visualization for a specified time period.

[0015] Compared with the prior art, the present invention has the following beneficial technical effects:

[0016] This invention solves the problem of preprocessing aircraft bus data. By preprocessing aircraft bus data and similar aircraft bus data, it enables rapid extraction of bus data by column. By constructing a data pyramid, it solves the problem of excessive data volume caused by loading raw high-frequency data at once during data visualization, as well as the resulting low visualization efficiency. It also enables hierarchical and rapid scheduling based on different resolution requirements. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of LOD sampling of column data in an embodiment of the present invention;

[0018] Figure 2 This is a sample of the original data;

[0019] Figures 3-5 This is the data after extracting by column;

[0020] Figures 6-9 A visual diagram illustrating different levels of data. Detailed Implementation

[0021] The present invention proposes an aviation bus data processing method based on a time pyramid, comprising the following steps:

[0022] S1. Clean and validate the data, removing invalid data. Figure 2The image shows sample raw data to be processed. The data is cleaned by loading the decoded data from the original flight bus data.

[0023] S2. Divide the complex table structure into columns, with each column containing a type of bus information data;

[0024] S3. Construct data combinations with time columns and data columns. This involves appending the time column to each data column to form a time-indexed data combination. The time column can be combined with multiple related data columns to form a single data type. For example, a time column can be combined with aircraft spatial location data, which consists of three columns containing the x, y, and z three-dimensional coordinate axes. The data format after column extraction is shown below. Figures 3-5 ;

[0025] S4. Take the combined time and data column data as the basic data to be processed, calculate the total duration of the data according to the start and end time intervals of the data, divide the data into n blocks according to the time, and then construct the bottom layer basic tile data of the time pyramid data. The number of data blocks can be defined by yourself.

[0026] S5. The basic data to be processed is thinned by halving the frequency each time, forming multi-level thinned data with decreasing frequency. After each successive thinning and block division, the data forms multi-level LOD data, such as... Figure 1 As shown;

[0027] For multi-level LOD data, besides the basic source data used for indicator analysis and data display, other levels of data are primarily used for user interaction and data display. When analyzing indicators, the system automatically analyzes algorithms or uses interactive methods to identify the start and end times of the data to be evaluated based on the specific events of the corresponding indicator. The evaluation system calculates the corresponding block segments of the source data to be scheduled based on the time period input by the user, thus achieving on-demand scheduling of the evaluation data. This design effectively avoids the problems of inaccurate evaluation results due to data thinning or the loading of large amounts of redundant data due to insufficient thinning.

[0028] The method for establishing the tile hierarchy uses 0 to 24 hours as the standard for tile segmentation throughout the entire time period. The top layer consists of two basic tiles, namely 0 to 12 hours and 12 to 24 hours, corresponding to tiles L0X0 and L0X1 respectively. This is then further subdivided downwards: L1X0, L1X1, L1X2, L1X3, and so on. When the subdivision reaches level 13, there are a total of 4096*2 tiles. Each tile covers a time of 10546.875ms. Assuming a maximum frequency of 100Hz, each tile contains approximately 1055 rows of data, which is a moderate amount of data. This method was initially chosen. The static correspondence between data frequency and its actual layer hierarchy is shown in Table 1 below.

[0029] Table 1. Static Correspondence between Data Frequency and its Actual Hierarchical Level

[0030] hierarchy Time period / tile (ms) Frequency range (Hz) 1 43200000 2 21600000 3 10800000 4 5400000 0.25 5 2700000 0.5 6 1350000 1 7 675000 2 8 337500 4 9 168750 8 10 84375 16 11 42187.5 32 12 21093.75 64 13 10546.875 128

[0031] S6. Divide the LOD data of each level into blocks according to time according to the method of constructing tiles for each level. Each data segment after being divided into blocks is called a data tile.

[0032] S7. Serialize the tile data into the database to form a hierarchical tile pyramid data set. This pyramid data serves as the foundation for visualization. During the visualization process, different levels of tile data can be called according to resolution requirements to achieve data visualization for a specified time period. For example... Figures 6-9 As shown, the vertical axis represents time, that is, the time from the start to the end of a flight, and the horizontal axis represents the normalized values ​​of various parameters at different time points.

[0033] This invention solves the problem of preprocessing aircraft bus data. By preprocessing aircraft bus data and similar aircraft bus data (relational table data that expresses various parameters using time as an index), it enables rapid extraction of bus data by column. By constructing a data pyramid, it solves the problem of excessive data volume caused by loading raw high-frequency data at once during data visualization, as well as the resulting low visualization efficiency. It also enables hierarchical and rapid scheduling based on different resolution requirements.

[0034] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited thereto. Various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of the present invention.

Claims

1. A method for processing aviation bus data based on a time pyramid, characterized in that, Includes the following steps: S1. Clean and validate the data, removing invalid data; S2. Divide the complex table structure into columns, with each column containing a type of bus information data; S3. Construct a data combination with a time column and a data column, that is, attach the time column to each data column to form a data combination indexed by time; S4. Take the combined time and data column data as the basic data to be processed, calculate the total duration of the data according to the start and end time intervals of the data, divide the data into n Block data segments according to the time, and then construct the bottom layer basic tile data of the time pyramid data. S5. The basic data to be processed is thinned by halving the frequency each time to form multi-level thinned data with decreasing frequency. After each thinning and block division, the data forms multi-level LOD data. S6. Divide the LOD data of each level into blocks according to time according to the method of constructing tiles for each level. Each data segment after being divided into blocks is called a data tile. S7. Serialize the tile data into the database to form a hierarchical tile pyramid data.

2. The aviation bus data processing method based on a time pyramid according to claim 1, characterized in that, In S3, a time column can be combined with multiple related data columns to form a single data category.

3. The aviation bus data processing method based on a time pyramid according to claim 1, characterized in that, In S4, the number of data blocks can be defined by the user.

4. The aviation bus data processing method based on a time pyramid according to claim 1, characterized in that, In S7, tile pyramid data serves as the foundation for visualization. During the visualization process, different levels of tile data can be called according to resolution requirements to achieve data visualization for a specified time period.

Images