Instrument data lightweight embedded database system

Abstract

The invention relates to a lightweight embedded database system for instrument data. The system comprises a header module, a mapping table module, a real-time data module and a data management module, wherein the header module is configured as a database header for storing the instrument data; the mapping table module is configured to store a database mapping table of instrument data; the real-time data module is configured as a database real-time data block for storing instrument data; the data management module is configured to manage a database header in a first management form, manage a database mapping table in a second management form, and manage a database real-time data block in a third management form. The instrument data lightweight embedded database system with the structure is light in storage mode, does not occupy too much space, is suitable for various verification and calibration systems conveniently, facilitates rapid storage of instrument data of the various verification and calibration systems, and has a data redundancy function and a function management design basis of safety and durability.

Description

Technical Field

[0001] This application relates to the technical field of verification and calibration systems, and particularly to a lightweight embedded database system for instrument data. Background Art

[0002] A verification and calibration system is a system used for the verification and calibration of measuring instruments and devices, aiming to ensure the accuracy and reliability of measurement results. Verification and calibration are two different activities, but they both involve the evaluation of the accuracy and reliability of measuring devices. Verification and calibration systems are widely used in various measuring instruments and devices, such as temperature sensors, pressure gauges, electronic scales, etc., and generally appear as a complete set of solutions for laboratories and industrial applications. Along with the verification and calibration system of the complete set of solutions, there are the processing requirements for a large amount of instrument data and the database system for instrument data.

[0003] In the current verification and calibration system, the verification and calibration system and the database system communicate through a serial port or a TCP port for instrument data interaction. Therefore, a large amount of instrument data is stored in the database system, and at the same time, it is necessary to ensure that the stored instrument data has a redundancy check and an exception recovery mechanism. If the instrument data is stored in the Flash space, the database system also needs to support the wear leveling function. When a system exception or a power supply exception occurs during the data operation process, the database system needs to have a write protection mechanism. Among them, traditional embedded databases such as the Sqlite3 database support large data storage, but do not support the recovery mechanism in case of data exceptions; the flashDB database supports large data storage, wear leveling, and data exception backup and recovery at the same time, but the database package reaches more than 10MB and cannot be ported and developed in a database with a small memory space, and the applicable verification and calibration systems are relatively limited. Summary of the Invention

[0004] In view of the above analysis, the embodiments of the present invention aim to provide a lightweight embedded database system for instrument data to solve the problem in the implementation results of the prior art that traditional embedded databases such as the Sqlite3 database support large data storage, but do not support the recovery mechanism in case of data exceptions. The flashDB database supports large data storage, wear leveling, and data exception backup and recovery at the same time, but the database package reaches more than 10MB and cannot be ported and developed in a database with a small memory space, and the applicable verification and calibration systems are relatively limited.

[0005] The lightweight embedded database system for instrument data in the embodiments of this application includes:

[0006] A header module, configured to store the database header of the instrument data;

[0007] A mapping table module, configured to store the database mapping table of the instrument data;

[0008] A real-time data module, configured to store a database real-time data block of the instrument data;

[0009] A data management module, configured to manage the database header in a first management form, manage the database mapping table in a second management form, and manage the database real-time data block in a third management form.

[0010] The lightweight embedded database system for instrument data according to an embodiment of the present application includes a header module, a mapping table module, a real-time data module, and a data management module. The header module is configured to store a database header of instrument data; the mapping table module is configured to store a database mapping table of instrument data; the real-time data module is configured to store a database real-time data block of instrument data; the data management module is configured to manage the database header in a first management form, manage the database mapping table in a second management form, and manage the database real-time data block in a third management form. The lightweight embedded database system for instrument data with the above architecture has a lightweight storage method, does not occupy too much space, is convenient for applying various calibration systems, and while facilitating the rapid storage of instrument data of various calibration systems, has a functional management design basis for data redundancy function, security, and durability.

[0011] As an optional embodiment, the database header is stored with a first set number of bytes; the database mapping table is stored with a second set number of bytes; the database real-time data block is stored with a third set number of bytes.

[0012] As an optional embodiment, the database header includes a database version number, a header storage space, the total number of data tables, and each data table;

[0013] Wherein, each data table includes a table name, a table real-time data length, the number of real-time entries, and the offset of the database mapping table in the flash space.

[0014] As an optional embodiment, the mapping table module stores a database mapping table in the form of a main mapping table and a database mapping table in the form of a slave mapping table;

[0015] The database mapping table includes the insertion status of the database real-time data block, the insertion serial number of the database real-time data block, the data length of the database real-time data block, and the offset of the database real-time data block in the flash space.

[0016] As an optional embodiment, the database real-time data block includes the serial number of the instrument data, the time of the instrument data, the type of the instrument data, the network signal of the instrument data, the content of the instrument data, the check code of the instrument data, and the data reporting status of the instrument data.

[0017] As one of the optional embodiments, the first set number of bytes, the second set number of bytes, and the third set number of bytes are all 4 kb.

[0018] As one of the optional embodiments, the process of managing the database header in the first management form includes:

[0019] Checking whether the database header is damaged through the unique identification code of the database version number, the threshold of the total number of data tables, and the header storage space;

[0020] Naming the table names of different data tables, and constructing different information record tables and table real-time data lengths according to the types of instrument data.

[0021] As one of the optional embodiments, managing the database mapping table in the second management form includes:

[0022] Managing the information of each inserted database real-time data block through a red-black tree, facilitating the upper-layer application to perform addition, deletion, update, and search operations on the database.

[0023] As one of the optional embodiments, the process of managing the database real-time data block in the third management form includes:

[0024] Encrypting the database real-time data block using the AES-CBC algorithm, and implementing encryption and decryption operations in the AES-ECB mode through the APIs provided by the mbed TLS library;

[0025] Performing content verification on the database real-time data block using the CRC16 algorithm;

[0026] The data content of the database real-time data block is filled in accordance with the MQTT protocol; wherein, the data format of the data content supports JSON and hex formats.

[0027] As one of the optional embodiments, the process of managing the database real-time data block in the third management form further includes:

[0028] Writing the database real-time data block into the flash, and dynamically writing to each sector in turn according to the offset of the database real-time data block in the flash space to balance the wear of each sector;

[0029] When the database real-time data block is about to be erased and rewritten, prohibiting flash operations when a soft restart is required through the system soft restart judgment mechanism. Description of the Drawings

[0030] Figure 1It is a structural diagram of a lightweight embedded database system module for instrument data in an application embodiment;

[0031] Figure 2 It is a database space layout diagram;

[0032] Figure 3 It is a flowchart of the data management module in an application embodiment;

[0033] Figure 4 It is a database operation flowchart. Specific implementation manners

[0034] In order to make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the described embodiments of the present application without creative efforts shall fall within the scope of protection of the present application.

[0035] Unless otherwise defined, the technical terms or scientific terms used in the present application shall have the ordinary meanings understood by those of ordinary skill in the art to which the present application pertains. The "first", "second", and similar terms used in the present application do not denote any order, quantity, or importance, but are only used to distinguish different components. The terms such as "including" or "comprising" mean that the elements or items appearing before this word cover the elements or items listed after this word and their equivalents, without excluding other elements or items. The terms such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms such as "upper", "lower", "left", "right", etc. are only used to represent relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0036] In order to keep the following description of the embodiments of the present application clear and concise, the detailed descriptions of some known functions and known components are omitted in the present application.

[0037] The embodiments of the present application provide a lightweight embedded database system for instrument data.

[0038] Figure 1 It is a structural diagram of a lightweight embedded database system module for instrument data in an application embodiment. As Figure 1 shown, a lightweight embedded database system for instrument data in an application embodiment includes:

[0039] A header module 100, configured to store the database header of instrument data;

[0040] The mapping table module 101 is configured to store the database mapping table of the instrument data;

[0041] The real-time data module 102 is configured to store the database real-time data block of the instrument data;

[0042] The data management module 103 is configured to manage the database header in a first management form, manage the database mapping table in a second management form, and manage the database real-time data block in a third management form.

[0043] Among them, the lightweight embedded database system for instrument data in the embodiment of the present application includes a basic database base, and the basic database base is used to store instrument data. The header module 100, the mapping table module 101, and the real-time data module 102 are subunits of the basic database, and the data management module 103 is independent of the basic database base and is configured to manage the basic database base.

[0044] In this embodiment, the basic database base is built based on a flash memory, and the flash space is planned and laid out, which is divided into a database header, a database mapping table, and a database real-time data block. Compared with the traditional database storage method, this structure has a very fast read and write speed and higher performance. Moreover, the storage method is lightweight and does not occupy too much space.

[0045] As one optional embodiment, the database header is stored with a first set number of bytes; the database mapping table is stored with a second set number of bytes; the database real-time data block is stored with a third set number of bytes.

[0046] Preferably, the first set number of bytes, the second set number of bytes, and the third set number of bytes are all 4096 bytes to support 4096-byte data storage, which is convenient for storage.

[0047] As one optional embodiment, Figure 2 is a database space layout diagram, as Figure 2 shown, the database header includes a database version number header storage space, the total number of data tables, and each data table;

[0048] Among them, each of the data tables includes a table name, a real-time data length of the table, a real-time number of records, and an offset of the database mapping table in the flash space. The mapping table module stores a database mapping table in the form of a main mapping table and a database mapping table in the form of a slave mapping table; the database mapping table includes an insertion status of the database real-time data block, an insertion serial number of the database real-time data block, a data length of the database real-time data block, and an offset of the database real-time data block in the flash space. The database real-time data block includes a serial number of the instrument data, a time of the instrument data, a type of the instrument data, a network signal of the instrument data, content of the instrument data, a check code of the instrument data, and a data reporting status of the instrument data.

[0049] Such as the database space layout Figure 2 As shown, taking the flash database space capacity of 8MB as an example, the database is divided into a database header, a mapping table of table N, and a real-time data block of table N, where 1 <= N <= 7. The database supports creating multiple tables, each table is independent, and each table has a main and a slave mapping table and a real-time data block.

[0050] Based on the database space layout Figure 2 , Figure 3 is a flowchart of the data management module for an application embodiment. As Figure 3 shown, the process of managing the database header in the first management form includes steps S10 and S11:

[0051] S10, verify whether the database header is damaged through the unique identification code of the database version number, the threshold of the total number of data tables, and the header storage space;

[0052] S11, name the table names of different data tables, and construct different information record tables and real-time data lengths of the tables according to the type of instrument data.

[0053] The second management form manages the database mapping table, including step S12:

[0054] S12, manage the information of each inserted database real-time data block through a red-black tree, facilitating the upper-layer application to perform addition, deletion, update, and search operations on the database.

[0055] The process of the third management form managing the database real-time data block includes steps S13 to S17:

[0056] S13, encrypt the database real-time data block using the AES-CBC algorithm, and implement encryption and decryption operations in the AES-ECB mode through the API provided by the mbed TLS library;

[0057] S14, perform content verification on the real-time data block of the database using the CRC16 algorithm;

[0058] S15, fill the data content of the real-time data block of the database according to the mqtt protocol; among them, the data format of the data content supports json and hex formats;

[0059] S16, write the real-time data block of the database into the flash, and dynamically write to each sector in turn according to the offset of the real-time data block of the database in the flash space to balance the wear of each sector;

[0060] S17, when the real-time data block of the database is about to be erased and rewritten, prohibit flash operations when a soft restart is required through the system soft restart judgment mechanism.

[0061] To explain steps S10 to S17, taking the first set number of bytes, the second set number of bytes, and the third set number of bytes in the preferred embodiment as 4 kb as an example, refer to Figure 4 shown in the database operation flowchart, the first management form of the database header specifically includes:

[0062] 1. The storage space occupies 4 KB and includes the database version number, the storage space of the header, the total number of data tables, and each data table.

[0063] 2. The header verifies whether the database header is damaged through the unique identification code of the database version number, the threshold of the total number of data tables, and the storage space of the header.

[0064] 3. The header supports multiple data tables and names different data table names. Each data table constructs different information record tables according to different verification and calibration systems. The real-time data storage sizes RealLength of different verification and calibration systems are different, and the number of real-time data storage records RealRecord is different. The calculation formula (1) of RealRecord is as follows:

[0065]

[0066]

[0067] Each table information content item in the header mainly includes the real-time data length RealLength of the table of the verification and calibration system, the maximum number of real-time data storage records RealRecord of the table, and the offsets tableOffset and tableBackupOffset of the database mapping table in the flash space.

[0068] Calculation formula (2): tableOffset = 0x1000 + 20 * i (0 <= i <= RealRecord)

[0069] The calculation formula (3) is obtained by using calculation formulas (1) and (2):

[0070] tableBackupOffset = (tableOffset + 4096 - 1) & (~(4096 - 1)) + 20 * j (0 <= j <= RealRecord, i = RealRecord).

[0071] Based on Figure 4 the process shown below, the second management form of the database mapping table is as follows:

[0072] 1. The mapping table manages the information of each piece of real-time data inserted through a red-black tree, facilitating the upper-layer application to perform addition, deletion, update, and search operations on the database.

[0073] 2. Each table has two tables, a main mapping table and a slave mapping table, which facilitates automatic recovery of the data table in case one of the mapping tables is damaged or abnormal during database insertion and deletion operations.

[0074] 3. The single-piece information content item in the mapping table mainly includes the real-time data insertion status, the real-time data length, and the offset itemOffset of the real-time data in the flash space.

[0075] The calculation formula (4) is obtained by using calculation formulas (1), (2), and (3):

[0076] itemOffset = (tableBackupOffset + 4096 - 1) & (~(4096 - 1)) + RealLength * k (0 <= k <= RealRecord, i = j = RealRecord).

[0077] Based on Figure 4 the process shown below, the third management form of the database real-time data block is as follows:

[0078] 1. The real-time data uses the AES-CBC algorithm to encrypt large data blocks, and the encryption and decryption operations in the AES-ECB mode can be implemented through the APIs provided by the mbed TLS library.

[0079] 2. The real-time data uses the CRC16 algorithm to verify the real-time data content.

[0080] 3. The real-time data supports a maximum of 4KB, the data content is filled in accordance with the mqtt protocol, and the data format supports json and hex formats.

[0081] 4. Write real-time data to flash. According to the written offset address itemOffset, dynamically write to each sector in turn to balance the wear of each sector.

[0082] 5. When real-time data is about to be erased and rewritten, add a system soft restart judgment mechanism. Once a soft restart is required, prohibit flash operations.

[0083] Figure 4 Under the database operation process of, the achievable advantages include:

[0084] 1. Fast. The read and write speed is very fast, and the performance is higher compared with the traditional database storage method.

[0085] 2. Lightweight. The storage method is lightweight and does not occupy too much space.

[0086] 3. Convenience. The storage method supports 4096-byte data storage, which is convenient for storage.

[0087] 4. Reliability. It has a data redundancy function and can automatically recover when data fails.

[0088] 5. Security. It has a data encryption function to avoid the risk of data leakage.

[0089] 6. Durability. It has wear leveling to extend the life of flash.

[0090] The lightweight embedded database system for instrument data in the embodiment of the present application includes a header module, a mapping table module, a real-time data module, and a data management module. The header module is configured to store the database header of instrument data; the mapping table module is configured to store the database mapping table of instrument data; the real-time data module is configured to store the database real-time data block of instrument data; the data management module is configured to manage the database header in a first management form, manage the database mapping table in a second management form, and manage the database real-time data block in a third management form. The lightweight embedded database system for instrument data with the above architecture has a lightweight storage method, does not occupy too much space, is convenient for applying various verification and calibration systems, and while facilitating the rapid storage of instrument data in various verification and calibration systems, has a functional management design basis with data redundancy function, security, and durability.

[0091] For the present application, the following points also need to be explained:

[0092] (1) The accompanying drawings of the embodiments of the present application only relate to the structures involved in the embodiments of the present application, and other structures can refer to the general design.

[0093] (2) For clarity, in the drawings used to describe the embodiments of the present invention, the thickness and dimensions of layers or structures are enlarged. It will be understood that when an element such as a layer, film, region, or substrate is referred to as being "on" or "under" another element, the element can be "directly" on or under the other element, or intervening elements may be present.

[0094] (3) Without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other to obtain new embodiments. The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. The protection scope of the present application shall be subject to the protection scope of the claims.

[0095] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0096] The above embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the patent application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. A lightweight embedded database system for instrument data, characterized in that: include: A header module, configured as a database header for storing instrument data; A mapping table module, configured to store a database mapping table of the meter data; A real-time data module configured to store a database real-time data block of the instrument data; The data management module is configured to manage the database header in a first management form, manage the database mapping table in a second management form, and manage the database real-time data block in a third management form.

2. The instrument data lightweight embedded database system according to claim 1 is characterized in that , the database header is stored with a first set number of bytes; the database mapping table is stored with a second set number of bytes; and the database real-time data block is stored with a third set number of bytes.

3. The instrument data lightweight embedded database system according to claim 1 is characterized in that ,The database header includes the database version number, header storage space, the total number of data tables and each data table; Each of the data tables includes a table name, a table real-time data length, a real-time number of entries, and an offset of the database mapping table in the flash space.

4. The instrument data lightweight embedded database system according to claim 1 is characterized in that , the mapping table module stores a database mapping table in the form of a master mapping table and a database mapping table in the form of a slave mapping table; The database mapping table includes the insertion status of the database real-time data block, the insertion index number of the database real-time data block, the data length of the database real-time data block and the offset of the database real-time data block in the flash space.

5. The instrument data lightweight embedded database system according to claim 4, characterized in that: The database real-time data block includes the serial number of the instrument data, the time of the instrument data, the type of the instrument data, the network signal of the instrument data, the content of the instrument data, the check code of the instrument data and the data reporting status of the instrument data.

6. The instrument data lightweight embedded database system according to claim 2, characterized in that: The first set byte number, the second set byte number and the third set byte number are all 4kb.

7. The instrument data lightweight embedded database system according to claim 3 is characterized in that The process of managing the database header in the first management form includes: Verify whether the database header is damaged by using the unique identification code of the database version number, the threshold of the total number of data tables, and the header storage space; Name the table names of different data tables, and construct different information recording tables and table real-time data lengths according to the type of instrument data.

8. The instrument data lightweight embedded database system according to claim 4, characterized in that: Managing the database mapping table in a second management form includes: The information of each real-time data block of the database inserted is managed by a red-black tree, so that upper-layer applications can add, delete, update and search the database.

9. The instrument data lightweight embedded database system according to claim 5, characterized in that: The process of managing the real-time data blocks of the database in the third management form includes: The database real-time data block is encrypted using the AES-CBC algorithm, and the encryption and decryption operations in the AES-ECB mode are implemented through the API provided by the mbed TLS library; Using CRC16 algorithm to verify the content of the real-time data block of the database; The data content of the real-time data block of the database is filled in accordance with the MQTT protocol; wherein the data format of the data content supports json and hex formats.

10. The instrument data lightweight embedded database system according to claim 5, characterized in that: The process of managing the real-time data blocks of the database in the third management form also includes: Writing the database real-time data block into the flash, and dynamically writing each sector in turn according to the offset of the database real-time data block in the flash space, so that each sector is worn evenly; When the real-time data block of the database is about to be erased, the flash operation is prohibited when a soft restart is required through the system soft restart judgment mechanism.

Images