Airborne system simulation experiment data recording system and design method thereof

Abstract

The invention provides an airborne system simulation experiment data recording system and a design method thereof, and the recording system comprises a data layer which is used for storing and managing simulation data and metadata; the core layer is used for providing flexible interfaces of different demand services for the core layer as the business layer, and the flexible interfaces comprise processing functions, plug-ins, scripts or query statements; and the business layer is used for differentially designing different functional modules according to different business requirements, and the functional modules comprise data stream access, data analysis, a signal data storage tool, a metadata storage tool, data processing and data playback. According to the airborne system simulation experiment data recording system and the design method thereof, recording, storage and calculation of data are realized, and efficient retrieval, query and batch processing are also realized; performance and time overhead brought by transmission of data among a storage system, a message queue and stream computing software can be reduced in a multifunctional set mode, and the data recording capacity of aviation airborne system simulation is improved.

Description

Technical Field

[0001] The present invention relates to numerical simulation experiments of airborne systems in aviation, and particularly to a data recording system for airborne system simulation experiments and a design method thereof. Background Art

[0002] With the rapid development of computer technology, digital simulation technology has become an important production tool. In the aerospace field, numerical simulation technology has been widely used and remarkable achievements have been made. In the simulation experiments of airborne systems in aviation, as the number of simulation nodes increases, the simulation models become more complex, and the simulation scale becomes larger, it directly leads to increasingly dense data in the simulation system, and the performance pressure on the recording software based on traditional databases becomes greater. In this regard, it is urgent to upgrade and improve the relevant data recording software to meet the expanding and upgrading requirements of the airborne simulation system in terms of form and scale.

[0003] Currently, the relevant research mainly focuses on two aspects: the underlying system and the software architecture. The research on the underlying system mainly focuses on how to improve the reading and writing efficiency of files, the caching mechanism, and the I / O throughput rate of files. The main means include using high-speed storage media, designing an efficient file storage system, and combining multi-level caching mechanisms. This method can only relieve the software performance pressure, and still cannot be effectively improved in terms of large data processing, recording, and storage. In terms of architecture, based on the Hadoop distributed HDFS file system and the HBASE distributed database, big data is distributed and stored on different computer nodes to solve the pressure brought by the storage bottleneck, and technologies such as Spark and MapReduce are used to distribute the data processing tasks to the computing nodes storing the data for processing, which can well solve the problems of a large amount of data transmission and limited technical capabilities of a single machine. These general big data technologies are widely used and can handle both structured data and unstructured data well. Although they can be directly applied to the airborne digital simulation system in aviation, they are not specifically optimized for the simulation data in the form of time-series sequences. In addition, the data storage capacity of the airborne digital simulation system in aviation does not reach a large order of magnitude, and the transfer of data between the storage system and the stream computing software brings relatively high performance and time overhead.

[0004] Therefore, it is necessary to provide a data recording system for airborne system simulation experiments and a design method for the system architecture to address the problem that the existing data recording and processing capabilities cannot meet the requirements. Summary of the Invention

[0005] To solve the problems in the prior art, the present invention provides an airborne system simulation experiment data recording system and its design method, which can achieve the purpose of fast retrieval at the data layer, improve the fast response ability of the storage system at the core processing layer, reduce the performance overhead of the hardware system, ensure the high-speed data writing ability of the entire system, facilitate the expansion of service requirements, and improve the recording efficiency; at the same time, it realizes the recording, storage, and calculation of data, and also realizes efficient retrieval, query, and batch processing; it can reduce the performance and time overhead caused by the transfer of data between the storage system, message queue, and stream computing software in a multi-functional manner, and enhance the data recording ability of the aviation airborne system simulation.

[0006] To achieve the above object, the present invention provides the following technical solutions: An airborne system simulation experiment data recording system includes: The data layer, located at the bottom layer of the recording system architecture, is used to store and manage simulation data and metadata, including simulation data and metadata. The simulation data is derived from simulation bus signals, and the metadata is used to describe the data packets of the simulation data. The core layer, based on the data layer, is used to improve the fast response ability of the data layer, including a memory table, message queue, storage engine, and calculation engine. The interface layer, located between the core layer and the service layer, provides a flexible interface for the core layer to serve different requirements of the service layer, including processing functions, plugins, scripts, or query statements. The service layer, located at the top layer of the recording system architecture, is used to design different functional modules according to different service requirements, including data stream access, data parsing, signal data storage tool, metadata storage tool, data processing, and data playback.

[0007] In a preferred technical solution, in the data layer, according to the characteristics of the aviation signal data, different attributes are distinguished, and tree structure classification management and storage are performed to achieve fast retrieval.

[0008] In a preferred technical solution, in the data layer, the simulation data and metadata of the simulation signal are stored in the form of a structured data table; the simulation data is stored in the signal data table in the structure of a timestamp, parameter field name, and parameter value; the metadata is stored in the metadata table in the structure of the sending time, receiving time, number of signals, and expected number of the time-sequenced data packets of the simulation data sent by the external simulation stream server.

[0009] In a preferred technical solution, both the signal data table and the metadata table are managed in a tree structure classification according to specific fields such as different simulation domains, aviation signal types, signal names, aviation equipment locations, and aviation equipment numbers.

[0010] In a preferred technical solution, the in-memory table and message queue in the core layer provide a cache space for the data stream, computing engine, and storage engine transmitted between the service layer and the data layer; the computing engine triggers the computing function based on the data stream, and uses a sliding window to calculate and process data; the storage engine performs fast data retrieval and high-speed reading and writing of the physical storage medium based on a multi-level cache mechanism.

[0011] In a preferred technical solution, the storage engine adopts a three-level cache mechanism. The real-time hot data is stored in the built-in in-memory table and message queue, the in-real-time data is stored in the SSD hard disk, and the historical cold data is stored in the mechanical hard disk; the storage and retrieval of real-time hot data, in-real-time data, and historical cold data are all automatically managed by the storage engine.

[0012] In a preferred technical solution, the computing engine is responsible for the computing tasks of the entire recording system and provides complex computing processing for the service layer; the in-memory table and message queue cache the real-time hot data that arrives in real time and perform batch processing.

[0013] In a preferred technical solution, the processing functions of the interface layer include aggregation functions based on time windows and user-defined operation functions; complex operation tasks or processing flows can be completed through the processing functions or plug-ins of the interface layer in series to meet the extended service requirements of the entire recording system.

[0014] In a preferred technical solution, the data playback is the playback of historical data based on time windows; the data processing includes operation functions for adding, deleting, modifying, and querying in data management, mean operation functions, and detrending operation functions; the data parsing, signal data storage tool, and metadata storage tool in the service layer classify the data stream to improve the processing efficiency of the recording system.

[0015] Another object of the present invention is to provide a design method for an airborne system simulation experiment data recording system, which includes the following steps: Step 1: First, divide the architecture of the airborne system simulation experiment data recording system into a data layer, a core layer, an interface layer, and a service layer in sequence; Among them, the data layer is located at the bottom layer of the architecture of the recording system and is used to store and manage simulation data and metadata, including simulation data and metadata. The simulation data is derived from simulation bus signals, and the metadata is used to describe the data packets of the simulation data; The core layer is based on the data layer and is used to improve the fast response ability of the data layer, including an in-memory table and message queue, a storage engine, and a computing engine; The interface layer is located between the core layer and the service layer and provides a flexible interface for the core layer to provide different service requirements for the service layer, including processing functions, plug-ins, scripts, or query statements; The business layer is located at the top layer of the architecture of the recording system and is used to design different functional modules according to different business requirements, including data stream access, data parsing, signal data storage tool, metadata storage tool, data processing, and data playback; Step 2: In the data layer, according to the characteristics of the aviation signal data, different attributes are distinguished, and tree structure classification management and storage are performed to achieve rapid retrieval of the recording system; Step 3: In the core layer, the storage engine adopts a three-level cache mechanism. Real-time hot data is stored in the built-in memory table and message queue, in-real-time data is stored in the SSD hard disk, and historical cold data is stored in the mechanical hard disk; the storage and retrieval of real-time hot data, in-real-time data, and historical cold data are all automatically managed by the storage engine, improving the rapid response ability of the data layer of the recording system; Step 4: In the interface layer, complex operation tasks or processing flows are completed by concatenating aggregation functions based on time windows, user-defined operation functions, or plugins, realizing the extended service requirements of the entire recording system; Step 5: In the business layer, different functional modules are designed according to different business requirements, and classification processing is performed using data parsing, signal data storage tool, metadata storage tool, data processing, and data playback, improving the processing efficiency of the recording system in the business layer.

[0016] Compared with the prior art, the beneficial effects of an airborne system simulation experiment data recording system and its design method of the present invention are: 1. In the data layer, according to the characteristics of the aviation signal data, different attributes are distinguished, and tree structure classification management and storage are performed, which can achieve the purpose of rapid retrieval.

[0017] 2. In the core layer, the storage engine adopts a three-level storage mechanism. Real-time hot data is stored in the built-in message queue or internal cache, in-real-time data is stored in the SSD hard disk, and a large amount of historical cold data is stored in the mechanical hard disk. The storage and retrieval of the three types of data all rely on the automatic management of the storage engine, which can improve the rapid response ability of the storage system. The computing tasks of the recording system are undertaken by the computing engine, which can provide complex computing tasks for the business layer and at the same time reduce the performance overhead of the recording system. The mechanism of the memory table and message queue can cache the data arriving in real time and perform batch processing, ensuring the high-speed writing ability of the recording system for data.

[0018] 3. In the interface layer, aggregation functions based on time windows and user-defined functions or plugins are supported, and related processing functions or plugins are concatenated to form complex operation tasks or processing flows, facilitating the extension of service requirements.

[0019] 4. The business layer differentiates different business requirements and designs modules with different functions, including a historical data playback module based on a time window, a processing tool module integrating common efficient operations, etc. It also classifies and processes data using a data parsing module, a metadata and signal data storage module, further improving the efficiency of the recording system at the business level.

[0020] 5. The present invention integrates functions such as classifying and storing specific aviation signal data, message queues, cache mechanisms, and stream computing. It is uniformly managed and scheduled by a storage and processing engine, which not only realizes data recording, storage, and calculation but also realizes efficient retrieval, query, and batch processing functions. The multi-functional set can reduce the performance and time overhead caused by data transfer between the storage system, message queue, and stream computing software.

[0021] In summary, the technical solution described in the present invention can well optimize the storage and data processing performance of the numerical simulation experiment of the airborne system, and further improve the data recording ability of the airborne system simulation. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the software architecture of a data recording system for an airborne system simulation experiment of the present invention; Figure 2 It is a schematic diagram of the architecture of the processing function of the interface layer of the present invention; Figure 3 It is a schematic diagram of the data processing architecture of the business layer of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0023] Refer to Figures 1-3 to further describe the architecture and design method of a data recording system for an airborne system simulation experiment of the present invention. Embodiment

[0024] As Figure 1 shown: A data recording system for an airborne system simulation experiment includes: The data layer, located at the bottom layer of the architecture of the recording system, is generally a local magnetic hard disk and a file management system, used for storing and managing simulation data and metadata, including simulation data and metadata. The simulation data is sourced from simulation bus signals, and the metadata is used to describe the data packets of the simulation data; At the data layer, the simulation data and metadata of the simulation signals are stored in the form of structured data tables; the simulation data is stored in the signal data table in the structure of timestamp, parameter field name, and parameter value; the metadata is stored in the metadata table in the structure of the sending time, receiving time, number of signals, and expected number of the time-sequenced simulation data packets sent by the external simulation stream server. Both the signal data table and the metadata table are managed and classified in a tree structure according to specific fields such as different simulation domains, aviation signal types, signal names, aviation equipment locations, and aviation equipment numbers. At the data layer, according to the characteristics of the aviation signal data, different attributes are distinguished, and tree-structured classification management and storage are carried out to achieve fast retrieval.

[0025] The core layer is above the data layer and is the core part of this software architecture, used to improve the fast response ability of the data layer, including a memory table, message queue, storage engine, and computing engine; the memory table and message queue in the core layer provide a cache space for the data stream, computing engine, and storage engine transmitted between the service layer and the data layer; the computing engine triggers the computing function based on the data stream and uses a sliding window to calculate and process the data; the storage engine performs fast retrieval of data and high-speed reading and writing of the physical storage medium based on a multi-level cache mechanism. The storage engine adopts a three-level cache mechanism, with real-time hot data stored in the built-in memory table and message queue, incoming real-time data saved in the SSD hard disk, and historical cold data stored in the mechanical hard disk; the storage and retrieval of real-time hot data, incoming real-time data, and historical cold data are all automatically managed by the storage engine. The computing engine is responsible for the computing tasks of the entire recording system and provides complex computing processing for the service layer; the memory table and message queue cache the real-time hot data that arrives in real-time and process it in batches.

[0026] The interface layer is between the core layer and the service layer and provides a flexible interface for the core layer to serve different requirements of the service layer, including processing functions, plugins, scripts, or query statements; the processing functions in the interface layer include aggregation functions based on time windows and user-defined operation functions; complex operation tasks or processing flows can be completed in series through the processing functions or plugins in the interface layer to meet the extended service requirements of the entire recording system.

[0027] The service layer is at the top layer of the recording system architecture and is used to design different functional modules according to different business requirements, including data stream access, data parsing, signal data storage tool, metadata storage tool, data processing, and data playback. The data playback is the historical data playback based on a time window; the data processing includes operation functions for adding, deleting, modifying, and querying in data management, mean operation functions, and detrending operation functions; the data parsing, signal data storage tool, and metadata storage tool in the service layer classify and process the data stream to improve the processing efficiency of the recording system.

[0028] The following further describes the present invention for a clearer description. In combination with the specific processes of data storage, data playback, and data processing of the recording system of the present invention, the present invention is further described.

[0029] (1) Data storage process The data storage process includes processes such as accessing the simulation data stream, parsing data, and storing data. The specific process is as follows: Process 1: The data stream access module at the service layer receives the timed data packet (i.e., the simulation data stream) from the external simulation stream server and passes the timed data packet to the data parsing module.

[0030] Process 2: The data parsing module parses information such as the simulation domain, aviation signal type, specific signal name, aviation equipment location or number, specific field values in the signal, data packet sending time and receiving timestamp, and the number of signals in the data packet, and further submits the parsed data to the signal data storage tool and metadata storage tool at the service layer.

[0031] Process 3: The signal data storage tool and metadata storage tool at the service layer add auxiliary information to the data and element data, convert it into the corresponding storage data structure, and then use the interface functions or plugins at the interface layer to submit the simulation data and metadata to the memory table and message queue at the core layer for further processing.

[0032] Process 4: When there is a storage task, the storage engine at the core layer is awakened, obtains the simulation data and metadata from the memory table and message queue. Based on the multi-level cache mechanism, the storage engine transfers the corresponding data from the memory to the solid-state drive or mechanical hard drive one by one in batches, and the simulation data and metadata are stored separately and managed in a tree structure in the file system, effectively improving the data retrieval efficiency.

[0033] (2) Data playback process The data playback function is based on the time sliding window mechanism. According to the rate of the accessed data stream, the historical data within a certain period of time is restored at the same rate into the same data stream. The specific process is as follows: Process 1: The data playback module initiates a historical data playback request, creates a playback task request through the processing function or plugin based on the time sliding window mechanism in the interface layer, and adds it to the playback task queue at the core layer.

[0034] Process 2: When there is a playback data task, the computing engine at the core layer is awakened, provides sliding window calculation for the historical data playback task, and creates a data reading task. The storage engine at the core layer retrieves and reads data according to the task request, then caches it in the data queue, and transfers the historical data to the data playback module at the service layer one by one in batches through the interface layer.

[0035] Process 3: The data playback module at the service layer plays back historical signal data according to the data reception rate.

[0036] (3) Data processing process The data processing tool at the service layer can provide common data management and processing functions, including functions for adding, deleting, modifying, and querying operations in data management, mean operation functions, detrending operation functions, etc., and also supports real-time and offline data processing. For the add, delete, modify, and query operations in data management, they are completed with the help of the storage engine module at the core layer, while operations such as mean or detrending also require the computing power provided by the computing engine module at the core layer to be completed. Among them, the operation process with the help of the computing engine at the core layer is as follows: Process 1: The data processing tool creates a task request and adds the data operation task to the task queue at the core layer through the interface layer.

[0037] Process 2: When there is an operation task, the computing engine at the core layer is awakened, creates a data acquisition task according to the operation task, and adds it to the task queue for data acquisition.

[0038] Process 3: The storage engine at the core layer retrieves and reads the corresponding data, and transfers the data to the computing engine for data calculation and processing.

[0039] Process 4: The computing engine at the core layer submits the calculated result to the data processing module at the service layer through the interface layer for display, completing the corresponding data processing process.

[0040] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. It should be noted that any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An airborne system simulation experiment data recording system, characterized in that: It includes: The data layer is located at the bottom layer of the architecture of the recording system and is used to store and manage simulation data and metadata, including simulation data and metadata, the simulation data is derived from the simulation bus signal, and the metadata is used to describe the data packet of the simulation data; The core layer is based on the data layer and is used to improve the rapid response capability of the data layer. It includes memory tables and message queues, storage engines, and computing engines. The interface layer is located between the core layer and the business layer. It provides the core layer with flexible interfaces for different service requirements, including processing functions, plug-ins, scripts or query statements. The business layer is located at the top of the recording system architecture and is used to differentiate and design different functional modules according to different business needs, including data stream access, data analysis, signal data storage tools, metadata storage tools, data processing, and data playback.

2. The airborne system simulation experiment data recording system according to claim 1, characterized in that: In the data layer, different attributes are distinguished according to the characteristics of aviation signal data, and tree-structured classification management and storage are performed to achieve rapid retrieval.

3. The airborne system simulation experiment data recording system according to claim 2, characterized in that: In the data layer, simulation data and metadata of simulation signals are stored in the form of structured data tables; the simulation data is stored in the signal data table in the structure of timestamp, parameter field name and parameter value; the metadata is stored in the metadata table in the structure of sending time, receiving time, signal quantity and expected quantity of data packets of timed simulation data sent by an external simulation stream server.

4. The airborne system simulation experiment data recording system according to claim 3 is characterized in that: The signal data table and metadata table are both classified and managed in a tree structure according to specific fields of different simulation domains, aviation signal types, signal names, aviation equipment locations, and aviation equipment numbers.

5. The airborne system simulation experiment data recording system according to claim 4, characterized in that: The memory table and message queue in the core layer provide cache space for the data flow, computing engine, and storage engine transmitted between the business layer and the data layer; the computing engine triggers the computing function based on the data flow and uses a sliding window to calculate and process data; the storage engine performs fast data retrieval and high-speed reading and writing of physical storage media based on a multi-level cache mechanism.

6. The airborne system simulation experiment data recording system according to claim 5, characterized in that: The storage engine adopts a three-level cache mechanism, real-time hot data is stored in the built-in memory table and message queue, real-time data is saved in the SSD hard disk, and historical cold data is stored in the mechanical hard disk; the storage and retrieval of real-time hot data, real-time data, and historical cold data are all automatically managed by the storage engine.

7. The airborne system simulation experiment data recording system according to claim 6, characterized in that: The computing engine is responsible for the computing tasks of the entire recording system and provides complex computing processing for the business layer; the memory table and message queue cache the real-time hot data that arrives in real time and processes it in batches.

8. The airborne system simulation experiment data recording system according to claim 7, characterized in that: The processing functions of the interface layer include aggregation functions based on time windows and user-defined operation functions. Complex operation tasks or processing flows can be completed in series through the processing functions or plug-ins of the interface layer to realize the extended service requirements of the entire recording system.

9. The airborne system simulation experiment data recording system according to claim 8, characterized in that: The data playback is the playback of historical data based on a time window; the data processing includes the add, delete, modify and query operation functions, the mean operation function and the detrending operation function in data management; the data analysis, signal data storage tool and metadata storage tool of the business layer classify the data stream to improve the processing efficiency of the recording system.

10. A method for designing an airborne system simulation experiment data recording system according to any one of claims 1 to 9, characterized in that: It includes the following steps: Step 1: First, the architecture of the airborne system simulation experiment data recording system is divided into a data layer, a core layer, an interface layer, and a business layer in sequence; Step 2: In the data layer, different attributes are distinguished according to the characteristics of the aviation signal data, and tree-structured classification management and storage are performed to realize rapid retrieval of the recording system; Step 3: In the core layer, the storage engine adopts a three-level cache mechanism, real-time hot data is stored in the built-in memory table and message queue, real-time data is stored in the SSD hard disk, and historical cold data is stored in the mechanical hard disk; the storage and retrieval of real-time hot data, real-time data, and historical cold data are all managed automatically by the storage engine to improve the rapid response capability of the data layer of the recording system; Step 4: At the interface layer, complex operation tasks or processing flows are completed by using aggregation functions based on time windows, user-defined operation functions or plug-ins in series to realize the extended service requirements of the entire recording system; Step five, at the business layer, different functional modules are designed according to different business needs, and classified processing is performed using data analysis, signal data storage tools, metadata storage tools, data processing, and data playback to improve the processing efficiency of the recording system at the business layer.

Images