Virtual experiment simulation data-oriented database architecture and data query method

Abstract

The invention belongs to the technical field of simulation data processing, and particularly relates to a database architecture oriented to virtual experiment simulation data and a data query method. The database architecture comprises an inner-layer structure and an outer-layer structure, the outer-layer structure forms an index layer of a database through permutation and combination information of physical quantities and timestamps and is stored in an SQL relational database, and the inner-layer structure is a data layer formed by gridding data information obtained through virtual experiment simulation calculation and is stored in an IoTDB database in a TsFile format. According to the method, the storage and query efficiency of the virtual experiment simulation data is improved.

Description

Technical Field

[0001] The present application belongs to the technical field of simulation data processing, and in particular relates to a database architecture and data query method for virtual experiment simulation data. Background Art

[0002] At each stage of the life cycle of civil aircraft development, a large number of building block physical and virtual experiments are required to comprehensively evaluate and verify the functions and performance of various aspects of the aircraft system. The data results obtained from virtual and physical experiments can be regarded as special data types containing three dimensions of physical quantity, time, and space. The data scales of the three dimensions are different and are closely related to the type of experiment.

[0003] Most of the physical experiment data are time series data, while the simulation data of virtual experiments contain both some physical quantities and time data, and a large amount of grid data or grid-type physical data such as stress, strain, and temperature associated with grid data. When establishing a database for management, it is necessary to consider the continuity of data in multiple dimensions, because when users use three-dimensional data, they will also access data based on the spatial continuity of the data (for example, access an edge, a face, or a body of a three-dimensional structure). From the perspective of data storage, in order to maximize the performance of data access, when storing data at the bottom layer, the grid data should also be organized according to the spatial continuity of the data. However, the data structure used in the bottom layer of the computer is still a one-dimensional array (the "multidimensional array" in the data structure is essentially a collection of multiple one-dimensional arrays). Therefore, when designing a storage strategy for high-dimensional grid data, it is inevitable to reduce the dimensionality of the data, and the dimensionality reduction process will inevitably destroy the spatial continuity of some high-dimensional data, thereby affecting the efficiency of data reading and access.

[0004] In terms of database selection, when SQL relational database and IoTDB database are directly used to store virtual experimental data associated with three-dimensional grids obtained by finite element simulation, a large amount of data will be accumulated in the index layer, seriously affecting the query efficiency. Therefore, a new database storage architecture and query method need to be designed for virtual experimental data. Summary of the invention

[0005] In order to solve the above problems, the present application provides a database architecture and data query method for virtual experiment simulation data, providing underlying management support for applications with high data dependence such as data twin experiments.

[0006] The first aspect of the present application provides a database architecture for virtual experiment simulation data, including an inner and outer layer structure. The outer layer structure is composed of the permutation and combination information of physical quantities and timestamps, which together constitute the index layer of the database and is stored in a SQL relational database. The inner layer structure is a data layer composed of gridded data information obtained by virtual experiment simulation calculations, which is stored in the IoTDB database and stored in the TsFile format.

[0007] Preferably, the data structure in the SQL relational database includes experimental project number, domain, physical quantity, timestamp and hash value, and the data is split into multiple sub-tables according to the experimental project number, domain and physical quantity type, and the corresponding hash value is found based on the physical quantity and timestamp as the retrieval conditions.

[0008] Preferably, the gridded data obtained by the virtual experiment simulation calculation is mapped onto the space curve, and the mapped data is stored in the IoTDB database according to the order of the space curve using the TsFile format.

[0009] Preferably, the space curve comprises a Hilbert curve, a Z-order space filling curve, an R-tree space filling curve or a KD Tree space filling curve.

[0010] Preferably, the optimal spatial curve is selected by calculating the query cost, wherein when calculating the query cost, a query test is performed on the same test gridded data organized by different spatial curves using a query pattern set to obtain a spatial curve with the minimum query cost.

[0011] Preferably, the query cost includes query time consumption.

[0012] Preferably, the query pattern set includes multiple query patterns, and each query pattern corresponds to a simulation experiment analysis task.

[0013] The second aspect of the present application provides a data query method for virtual experiment simulation data, using the above-mentioned database for virtual experiment simulation data to query the test simulation data, the data query method comprising: Step S1: Perform a first query in the SQL relational database based on the physical quantity and timestamp in the user data query request to obtain a hash value; Step S2: Based on the hash value and the grid ID in the user data query request, a second query is performed in the IoTDB database to obtain the grid data.

[0014] This application improves the storage and query efficiency of virtual experiment simulation data. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a database architecture diagram of a preferred embodiment of the database architecture of the present application for virtual experiment simulation data.

[0016] Figure 2 This is a diagram of the data structure based on the Polystore architecture.

[0017] Figure 3 It is a schematic diagram of the three-dimensional filling of the Hilbert curve.

[0018] Figure 4 It is a flow chart of a preferred embodiment of the data query method for virtual experiment simulation data of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical scheme and advantages of the implementation of this application clearer, the technical scheme in the implementation of this application will be described in more detail in combination with the drawings in the implementation of this application. In the drawings, the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The described implementation is a part of the implementation of this application, not all of the implementations. The implementation described below with reference to the drawings is exemplary and is intended to be used to explain this application, and cannot be understood as a limitation on this application. Based on the implementation in this application, all other implementations obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. The implementation of this application is described in detail below in combination with the drawings.

[0020] The first aspect of the present application provides a database architecture for virtual experiment simulation data, such as Figure 1 As shown, it includes two layers of structure, the outer layer structure is composed of the permutation and combination information of physical quantities and timestamps, which together constitute the index layer of the database and are stored in the SQL relational database. The inner layer structure is a data layer composed of grid data information obtained by virtual experiment simulation calculations, which is stored in the IoTDB database in TsFile format.

[0021] This application builds a database architecture through a two-layer structure. The outer layer is a relational database. The hash value is queried based on the physical quantity and timestamp, and then the hash value is used to retrieve the corresponding grid data in the inner IoTDB database.

[0022] In some optional embodiments, the data structure in the SQL relational database includes experimental project number, domain, physical quantity, timestamp and hash value. The data is split into multiple sub-tables according to the experimental project number, domain and physical quantity type, and the corresponding hash value is searched based on the physical quantity and timestamp as retrieval conditions.

[0023] like Figure 2As shown, this embodiment introduces a multi-model data management technology based on Polystore, and uses a strategy of sub-library and sub-table to split the data into several sub-tables according to features such as experimental project number and physical quantity type, thereby improving query efficiency.

[0024] In some optional implementations, the gridded data obtained by the virtual experiment simulation calculation is mapped to a space curve, and the mapped data is stored in the IoTDB database according to the order of the space curve using the TsFile format.

[0025] In order to solve the problem of continuity destruction caused by data dimensionality reduction and compression for grid data stored in the IoTDB database, this embodiment uses spatial curves to densely arrange the high-dimensional (two-dimensional, three-dimensional) space of the entire model, maps the grid data points to the spatial curves one by one, and uses the TsFile format to store the mapped data according to the curve sequence, thereby achieving the goal of storing the simulation data structure of the high-dimensional space with one-dimensional data, and improving the efficiency of the IoTDB database in storing large-scale non-time series data.

[0026] In some optional implementations, the space curve includes a Hilbert curve, a Z-order space filling curve, an R-tree space filling curve, or a KD Tree space filling curve.

[0027] In some optional implementations, the optimal spatial curve is selected by calculating the query cost, wherein when calculating the query cost, a query test is performed on the same test gridded data organized by different spatial curves using a query pattern set to obtain the spatial curve with the minimum query cost.

[0028] This embodiment manages data by determining an appropriate space curve through the following steps.

[0029] Given a set of space-filling curves C={c1,c2,…,cm} and a three-dimensional simulation data file D, we can generate a one-dimensional data pattern for each space-filling curve after reducing the dimension of the three-dimensional simulation data file. This process is recorded as Si=SFC(D,ci). Then, for the three-dimensional simulation data file D, m one-dimensional storage strategies can be generated.

[0030] In order to measure the performance of these strategies, combined with the actual application needs of the data, conclusions are drawn through experimental tests. The access pattern of the simulation data file D is formalized. For the simulation data file D, a set of query patterns needs to be given. .

[0031] For each query mode, there will be a different query cost on each data storage mode Si. , then the entire model selection problem can be formalized as follows: .

[0032] Based on the above formula, the sample data is organized using different spatial curves and stored in IoTDB. Then, the query costs under various query modes are tested to select the most appropriate storage format for simulation experiment data and select appropriate filling curves and dimensionality reduction models for simulation high-dimensional experimental data.

[0033] Repeating the above steps, it is found that for stress simulation experimental data in three-dimensional space, using Hilbert curve for dimensionality reduction is a more reasonable storage solution, such as Figure 3 shown.

[0034] In some optional implementations, the query cost includes query time consumption. In alternative implementations, the query cost may also be resource consumption, space overhead, and the like.

[0035] In some optional implementations, the query pattern set includes multiple query patterns, and each query pattern corresponds to a simulation experiment analysis task.

[0036] A second aspect of the present application provides a data query method for virtual experiment simulation data, using a database for virtual experiment simulation data as described in any one of the above items to query the test simulation data, the data query method comprising: Step S1: Perform a first query in the SQL relational database based on the physical quantity and timestamp in the user data query request to obtain a hash value; Step S2: Based on the hash value and the grid ID in the user data query request, a second query is performed in the IoTDB database to obtain the grid data.

[0037] This application aims at establishing a unified management system for virtual experiment simulation data based on the established database architecture, and queries the data in the SQL database. First, the user data query request query is decomposed into sub-queries for the SQL relational database and the IoTDB database; the query is executed in the relational database to filter out the hash value corresponding to the data; the hash value is used as the query condition to quickly locate the TsFile of the grid data storage in IoTDB, so as to find the corresponding high-dimensional grid. Figure 4In the case shown, the user data query request is first determined, for example, the data of the physical quantity P_i at the time t_i and the grid id 1-100 in the experiment with query ID i, and then the above request is decomposed into relational query and time series query. The relational query includes entering the instruction "SELECT hash value FROM test data WHERE ID=i AND time=t_i AND physical quantity=P_i" in the SQL relational database. This instruction is used to select the hash value in the SQL relational database "test data" and assign this hash value to R. The time series query includes entering the instruction "SELECT hash value=R WHERE t_id=1-100" in the IoTDB database. This instruction is used to query the corresponding grid data according to the hash value and the specific grid id.

[0038] After testing, the data query test results established by this application are compared with the MySQL database as shown below.

[0039] (1) For single sensor / grid query, the simulation experiment data query time of this application is 4.68s, and the simulation experiment data query time using MySQL is 5.97s.

[0040] (2) For multi-sensor / grid query, the simulation experiment data query time of this application is 14.23s, and the simulation experiment data query time using MySQL is 15.27s.

[0041] (3) For timestamp / grid query, the simulation experiment data query time of this application is 11.49s, and the simulation experiment data query time using MySQL is 21.95s.

[0042] (4) For small-scale queries based on the spatial distribution of sensors, the simulation experiment data query time of this application is 13.58s, and the simulation experiment data query time using MySQL is 23.35s.

[0043] (5) For large-scale queries based on spatial distribution of sensors, the simulation experiment data query time of this application is 15.78s, and the simulation experiment data query time using MySQL is 342.59s.

[0044] It can be seen that the database structure and query method proposed in this application are suitable for the storage and query of large-scale virtual experiment data, and significantly improve the query efficiency of virtual experiment data.

[0045] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A database architecture for virtual experiment simulation data, characterized in that: It includes two layers, the outer layer structure, which is composed of the permutation and combination information of physical quantities and timestamps and constitutes the index layer of the database, and is stored in the SQL relational database. The inner layer structure is a data layer composed of grid data information obtained by virtual experiment simulation calculation, which is stored in the IoTDB database in TsFile format.

2. The database architecture for virtual experiment simulation data according to claim 1, characterized in that: The data structure in the SQL relational database includes experimental project number, domain, physical quantity, timestamp and hash value. The data is split into multiple sub-tables according to the experimental project number, domain and physical quantity type, and the corresponding hash value is found based on the physical quantity and timestamp as the search conditions.

3. The database architecture for virtual experiment simulation data according to claim 1, characterized in that: The gridded data obtained by the virtual experiment simulation calculation is mapped onto the space curve, and the mapped data is stored in the IoTDB database according to the order of the space curve using the TsFile format.

4. The database architecture for virtual experiment simulation data according to claim 3, characterized in that: The space curve includes a Hilbert curve, a Z-order space filling curve, an R-tree space filling curve or a KD Tree space filling curve.

5. The database architecture for virtual experiment simulation data according to claim 3, characterized in that: The optimal spatial curve is selected by calculating the query cost. When calculating the query cost, a query pattern set is used to perform query tests on the same test gridded data organized by different spatial curves to obtain the spatial curve with the minimum query cost.

6. The database architecture for virtual experiment simulation data according to claim 5, characterized in that: The query cost includes query time.

7. The database architecture for virtual experiment simulation data according to claim 5, characterized in that: The query pattern set includes multiple query patterns, and each query pattern corresponds to a simulation experiment analysis task.

8. A data query method for virtual experiment simulation data, characterized in that: Using the database for virtual experiment simulation data as described in any one of claims 1 to 7 to query the experiment simulation data, the data query method includes: Step S1: Perform a first query in the SQL relational database based on the physical quantity and timestamp in the user data query request to obtain a hash value; Step S2: Based on the hash value and the grid ID in the user data query request, a second query is performed in the IoTDB database to obtain the grid data.

Images