An embedded lightweight database system

Abstract

The application discloses an embedded light-weight database system. The application comprises a database file and management software. The database file is composed of a file header and storage units, each of which is composed of user data and management data. The file header comprises a database identifier, and the management data of the storage units comprises a data state value, a data check value, a user data length and a key value. The database system uses an allocation record table and a recovery record table to manage the database file, the allocation record table records the key value and offset address of the queried storage unit, and the recovery record table records the length and offset address of the recovered storage unit. The cache area of the database system comprises a plurality of data blocks, the cache area is formed by reading the storage units from the database file, and the cache area is managed through a flush value. The functions realized by the management software include querying data, deleting data and adding data. The application has the advantages of small FLASH storage occupation, small memory consumption, simple management data and convenient use.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, in particular to the field of embedded technology, and specifically relates to an embedded lightweight database system. Background Art

[0002] A database system (DBS) is a system composed of database files and management software. A database is a warehouse that organizes, stores, and manages data according to data structures.

[0003] Embedded systems are special-purpose computer systems based on computer technology, with customizable hardware and software to meet the strict requirements of application systems on functionality, reliability, cost, power consumption, etc.

[0004] In computer applications, databases are essential for data management due to the constant need for data manipulation. While these data management systems are sophisticated and feature-rich, they are inappropriate for many embedded devices. First, these database management software consumes a large amount of storage space. Second, the programs rely on numerous dependencies. Finally, the software execution is complex, placing high demands on computer chips. Embedded devices have limited hardware resources, with limited Flash memory and RAM. The operating frequency of embedded chips is limited, making most databases inoperable or performing poorly in these devices. However, embedded systems also require data processing. Without data management, development time and project cycles would be lengthy. Therefore, a lightweight database is needed that provides basic database functionality while occupying minimal resources and being easy to use. Summary of the Invention

[0005] The purpose of the present invention is to provide an embedded lightweight database system.

[0006] The present invention includes one or more database files, and management software.

[0007] The database file is composed of a file header DB_Header and multiple storage units DB_Store_Unit. Each storage unit is composed of user data userdata and management data DB_record. User data is the actual stored data, and management data is the data created by the database to manage user data.

[0008] The file header DB_Header includes a 4-byte database identifier magic.

[0009] The management data of the storage unit DB_Store_Unit consists of data state value Ds, data check value Dc, user data length Dl and key value Kv, and has a length of 9 bytes; the data state value Ds has two flags of existence and deletion; the key value is an index value for searching the management data, and the key value of each storage unit is unique.

[0010] The database system uses allocation record table and recovery record table to manage the database files. The allocation record table records the key value Kv and offset address offset of the searched storage unit; the offset address offset is the length from the start position 0 of the database file to the start position of the management data of the storage unit. The recovery record table records the length len and offset address offset of the recovered storage unit. The allocation record table and the recovery record table run in the memory, and are created by reading each storage unit of the database file in sequence when the database system is initialized. The cache area of the database system is the data of part of the database files in the memory, and the cache area includes multiple data blocks, and the storage content of the data block is the storage content of the storage unit; the cache area is formed by reading the storage unit from the database file when the database system is initialized, and the flush value cache manages the cache area, and the flush value cache has two flags of flushing and not flushing.

[0011] The functions realized by the management software include searching data, deleting data and adding data. The specific implementation method is as follows:

[0012] Step (1) starts the database system, and the management software detects whether the database file exists; if not, the database file is created, the file header DB_Header is written, and step (2) is executed; if yes, step (2) is directly executed;

[0013] Step (2) opens the database file, reads the file header DB_Header; if the file header DB_Header is damaged, an error is reported, and the program is terminated; if the file header DB_Header is normal, the management data of the storage unit DB_Store_Unit is read, and step (3) is executed;

[0014] Step (3) reads the data state value Ds of the management data of each storage unit in sequence; if the data state value Ds is marked as existing, the key value Kv and offset address offset of the management data are written to the allocation record table; if the data state value Ds is marked as deleted, the storage unit length len and offset address offset of the management data are written to the recovery record table; thereby the allocation record table and the recovery record table are established;

[0015] Step (4) management software monitors user instruction: if the user instruction is query, execute steps (5)-(6); if the user instruction is delete, execute steps (7)-(8); if the user instruction is add, execute steps (9)-(11);

[0016] Step (5) queries the allocation record table according to the key value Kv of the data required to be queried by the user;

[0017] If the key value Kv exists in the allocation record table, read the data block of the cache area according to the offset address offset corresponding to the key value Kv, read the user data according to the user data length Dl recorded by the management data of the data block, and if the reading is successful, the query process is completed, and if the user data is not read in the cache area, execute step (6);

[0018] If the key value Kv does not exist in the allocation record table, report that the data does not exist, and the query process is completed;

[0019] Step (6) judges the state of the refresh value cache;

[0020] If the refresh value cache is a refresh flag, erase the data of the corresponding storage unit according to the start and end addresses of the cache area, and write the cache area data into the corresponding storage unit; read the storage unit in the database file according to the offset address offset corresponding to the key value Kv, read the user data according to the user data length Dl recorded by the management data in the storage unit, and if the reading is successful, the query process is completed;

[0021] If the refresh value cache is a non-refresh flag, read the storage unit in the database file according to the offset address offset corresponding to the key value Kv, read the user data according to the user data length Dl recorded by the management data in the storage unit, and if the reading is successful, the query process is completed;

[0022] Step (7) queries the allocation record table according to the key value Kv of the data required to be deleted by the user;

[0023] If the key value Kv exists in the allocation record table, read the data state value Ds of the management data of the data block of the cache area according to the offset address offset corresponding to the key value Kv; if the reading is successful, set the data state value Ds of the data block of the cache area to a deletion flag, set the refresh value cache to a refresh flag, and execute step (8); if the reading is unsuccessful, directly execute step (8);

[0024] If the key value Kv does not exist in the allocation record table, report that the data does not exist, and the deletion process is completed;

[0025] Step (8) judges the state of the refresh value cache;

[0026] If the refresh value cache is a refresh flag, the data of the corresponding storage unit is erased according to the start and end addresses of the cache area, and the cache area data is written to the corresponding storage unit; then, according to the offset address offset corresponding to the key value Kv, the data of the cache area length is read from the database file into the cache area, and the data status value Ds of the corresponding data block is set to a deletion flag; the offset address offset corresponding to the key value Kv and the storage unit length len corresponding to the offset address offset are written into the recycling record table, and the key value Kv and the corresponding offset address offset are deleted from the allocation record table, and the deletion process is completed;

[0027] If the refresh value cache is set to the non-refresh flag, read the data of the cache length from the database file according to the offset address offset corresponding to the key value Kv into the cache, set the data status value Ds of the corresponding data block to the delete flag, and set the refresh value cache to the refresh flag; write the offset address offset corresponding to the key value Kv and the storage unit length len corresponding to the offset address offset into the recycling record table, delete the key value Kv and the corresponding offset address offset from the allocation record table, and the deletion process is completed;

[0028] Step (9) searches the recycling record table for a storage unit length len that is the same as the length of the data to be added by the user plus the management data length of 9 bytes;

[0029] If the storage unit length len exists, find the data block corresponding to the offset address offset in the cache area according to the offset address offset corresponding to the storage unit length len, set the data status value Ds of the data block as the existence flag, write the data check value Dc and key value Kv of the data to be added; erase the user data userdata of the data block, write the data to be added in the user data area, and set the refresh value cache to the refresh flag; execute step (10);

[0030] If the storage unit length len does not exist, execute step (11);

[0031] Step (10) writes the offset address offset and key value Kv into the allocation record table, deletes the offset address offset and the corresponding storage unit length len in the recycling record table, and the adding process is completed;

[0032] Step (11) determines the status of the refresh value cache;

[0033] If the refresh value cache is not the refresh flag, a data block is newly created in the cache area according to the maximum offset address offset of the allocation record table, the data state value Ds of the data block is set to exist, the data check value Dc, the user data length Dl and the key value Kv of the data to be added are written, and the data to be added is written in the user data area; the refresh value cache is set to refresh; the offset address offset and the key value Kv are written into the allocation record table, and the adding process is completed.

[0034] If the refresh value cache is the refresh flag, the data of the corresponding storage unit is erased according to the start and end addresses of the cache area, and the cache area data is written into the corresponding storage unit; a data block is newly created in the cache area according to the maximum offset address offset of the allocation record table, the data state value Ds of the data block is set to exist, the data check value Dc, the user data length Dl and the key value Kv of the data to be added are written, and the data to be added is written in the user data area; the offset address offset and the key value Kv are written into the allocation record table, and the adding process is completed.

[0035] The database system of the application can realize the basic functions of data management, including querying data, deleting data and adding data. In the application, the memory occupation is mainly distributed in the buffer area, the allocation record table and the recovery record table. The allocation record table and the recovery record table dynamically allocate memory, and the memory consumption is extremely small. When operating data, only the allocation record table of the memory needs to be queried, if the offset value hits the cache area, the memory data is directly operated, and the operation speed is fast.

[0036] The database system of the application has the characteristics of small FLASH storage occupation, low requirement, small memory consumption, simple data management and convenient use. The light-weight database is used to manage and store data, which is convenient for software development, shortens the development cycle of the project, occupies less resources and saves costs. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 It is a single database file structure schematic diagram;

[0038] Figure 2 It is a cache area schematic diagram of the database system. DETAILED DESCRIPTION

[0039] An embedded light-weight database system, including one or more database files and management software.

[0040] The database file is composed of a file header DB_Header and a plurality of storage units DB_Store_Unit. Each storage unit is composed of user data userdata and management data DB_record. The user data is the actual stored data, and the management data is the data created by the database for managing the user data. The structure is as followsFigure 1 shown.

[0041] The file header DB_Header includes a 4-byte database identifier magic.

[0042] The management data of the storage unit DB_Store_Unit consists of the data status value Ds, data check value Dc, user data length Dl and key value Kv, with a length of 9 bytes; the data status value Ds has two flags: existence and deletion; the key value is the index value for searching management data, and the key value of each storage unit is unique.

[0043] The database system manages database files using an allocation log table and a reclaim log table. The allocation log table records the key value (Kv) and offset address (offset) of each queried storage unit. The offset address (offset) is the length from the database file's start position 0 to the start of the storage unit's management data. The reclaim log table records the length (len) and offset address (offset) of each reclaimed storage unit. The allocation and reclaim log tables run in memory and are created during database system initialization by sequentially reading the storage units in each database file.

[0044] like Figure 2 As shown, the database system's cache contains data from a portion of the database file in memory. The cache consists of multiple data blocks, each containing the same storage unit. During database system initialization, the cache is created by reading storage units from the database file. A refresh value cache manages the cache, with two flags: refresh and non-refresh. The cache is used for two purposes: first, to speed up reads and writes, as memory reads and writes are much faster than flash. Second, it reduces the number of flash writes, extending the life of the flash.

[0045] The functions implemented by the management software include querying data, deleting data, and adding data. The specific implementation methods are as follows:

[0046] Step (1) starts the database system, and the management software detects whether the database file exists; if it does not exist, creates the database file, writes the file header DB_Header, and executes step (2); if it exists, directly executes step (2);

[0047] Step (2) opens the database file and reads the file header DB_Header; if the file header DB_Header is damaged, reports an error and terminates the program; if the file header DB_Header is normal, reads the management data of the storage unit DB_Store_Unit and executes step (3);

[0048] Step (3) reads the data state value Ds of the management data of each storage unit in turn; if the data state value Ds is marked as existing, writes the key value Kv and the offset address offset of the management data to the allocation record table; if the data state value Ds is marked as deleted, writes the storage unit length len and the offset address offset of the management data to the recovery record table; thereby establishing the allocation record table and the recovery record table;

[0049] Step (4) the management software monitors the user instruction: if the user instruction is query, executes steps (5)-(6); if the user instruction is delete, executes steps (7)-(8); if the user instruction is add, executes steps (9)-(11);

[0050] Step (5) queries the allocation record table according to the key value Kv of the data to be queried by the user;

[0051] If the key value Kv exists in the allocation record table, reads the data block of the cache area according to the offset address offset corresponding to the key value Kv, reads the user data according to the user data length Dl recorded by the management data of the data block, and if the reading is successful, the query process is completed, and if the user data is not read in the cache area, step (6) is executed;

[0052] If the key value Kv does not exist in the allocation record table, reports that the data does not exist, and the query process is completed;

[0053] Step (6) judges the state of the flush value cache;

[0054] If the flush value cache is a flush flag, erases the data of the corresponding storage unit according to the start and end addresses of the cache area, and writes the data of the cache area into the corresponding storage unit; reads the storage unit in the database file according to the offset address offset corresponding to the key value Kv, reads the user data according to the user data length Dl recorded by the management data in the storage unit, and if the reading is successful, the query process is completed;

[0055] If the flush value cache is a non-refresh flag, reads the storage unit in the database file according to the offset address offset corresponding to the key value Kv, reads the user data according to the user data length Dl recorded by the management data in the storage unit, and if the reading is successful, the query process is completed;

[0056] Step (7) queries the allocation record table according to the key value Kv of the data to be deleted by the user;

[0057] If the key value Kv exists in the allocation record table, the data state value Ds of the management data of the data block of the cache area is read according to the offset address offset corresponding to the key value Kv; if the reading is successful, the data state value Ds of the data block of the cache area is set as a deletion flag, the flush value cache is set as a flush flag, and step (8) is executed; if the reading is unsuccessful, step (8) is directly executed;

[0058] If the key value Kv does not exist in the allocation record table, it is reported that the data does not exist, and the deletion process is ended;

[0059] Step (8) judges the state of the flush value cache;

[0060] If the flush value cache is a flush flag, the data of the corresponding storage unit is erased according to the start and end addresses of the cache area, and the cache area data is written into the corresponding storage unit; then, the data of the cache area length is read from the database file according to the offset address offset corresponding to the key value Kv, and the data state value Ds of the corresponding data block is set as a deletion flag; the offset address offset corresponding to the key value Kv and the storage unit length len corresponding to the offset address offset are written into the recovery record table, the key value Kv and the corresponding offset address offset are deleted from the allocation record table, and the deletion process is completed;

[0061] If the flush value cache is a non-flush flag, the data of the cache area length is read from the database file according to the offset address offset corresponding to the key value Kv, and the data state value Ds of the corresponding data block is set as a deletion flag; the flush value cache is set as a flush flag; the offset address offset corresponding to the key value Kv and the storage unit length len corresponding to the offset address offset are written into the recovery record table, the key value Kv and the corresponding offset address offset are deleted from the allocation record table, and the deletion process is completed;

[0062] Step (9) finds the storage unit length len same as the length of the data to be added plus 9 bytes of management data in the recovery record table;

[0063] If the storage unit length len exists, the data block of the offset address offset is found in the cache area according to the offset address offset corresponding to the storage unit length len, the data state value Ds of the data block is set as an existing flag, the data check value Dc and the key value Kv of the data to be added are written; the user data userdata of the data block is erased, the data to be added is written in the user data area, the flush value cache is set as a flush flag; and step (10) is executed;

[0064] If the storage unit length len does not exist, step (11) is executed;

[0065] Step (10) writes the offset address offset and the key value Kv into the allocation record table, deletes the offset address offset and the corresponding storage unit length len in the recycle record table, and adds the flow completion;

[0066] Step (11) judges the state of the refresh value cache;

[0067] If the refresh value cache is the non-refresh flag, a data block is newly created in the cache area according to the maximum offset address offset in the allocation record table, the data state value Ds of the data block is set as existing, the data check value Dc, the user data length Dl and the key value Kv of the data to be added are written, and the data to be added is written in the user data area; the refresh value cache is set as refreshing; the offset address offset and the key value Kv are written into the allocation record table, and the flow completion is added;

[0068] If the refresh value cache is the refreshing flag, the data of the corresponding storage unit is erased according to the start and end addresses of the cache area, and the cache area data is written into the corresponding storage unit; a data block is newly created in the cache area according to the maximum offset address offset in the allocation record table, the data state value Ds of the data block is set as existing, the data check value Dc, the user data length Dl and the key value Kv of the data to be added are written, and the data to be added is written in the user data area; the offset address offset and the key value Kv are written into the allocation record table, and the flow completion is added.

Claims

1. An embedded lightweight database system, comprising one or more database files and management software; characterized in that: the database file is composed of a file header DB_Header and a plurality of storage units DB_Store_Unit, each storage unit is composed of user data userdata and management data DB_record; the file header DB_Header comprises a database identifier magic; the management data of the storage unit DB_Store_Unit is composed of a data state value Ds, a data check value Dc, a user data length Dl and a key value Kv; the data state value Ds has two flags of existence and deletion; the key value is an index value for searching the management data, and the key value of each storage unit is unique; the database system uses an allocation record table and a recovery record table to manage the database file; the allocation record table records the key value Kv and the offset address offset of the searched storage unit; the offset address offset is the length from the starting position 0 of the database file to the starting position of the management data of the storage unit; the recovery record table records the length len and the offset address offset of the recovered storage unit; the allocation record table and the recovery record table run in the memory, and are created by reading each storage unit of the database file in sequence during initialization of the database system; the cache area of the database system is part of the data of the database file in the memory, and the cache area comprises a plurality of data blocks, and the storage content of the data block is the storage content of the storage unit; during initialization of the database system, the cache area is formed by reading the storage unit from the database file, and the flush value cache manages the cache area, and the flush value cache has two flags of flushing and not flushing; the management software realizes functions including searching data, deleting data and adding data.

2. The embedded lightweight database system of claim 1, wherein: The specific implementation method of the management software is as follows: Step (1) start the database system, and the management software detects whether the database file exists; if not, create the database file, write the file header DB_Header, and execute step (2); if yes, directly execute step (2); Step (2) open the database file, read the file header DB_Header; if the file header DB_Header is damaged, report an error, and the program terminates; if the file header DB_Header is normal, read the management data of the storage unit DB_Store_Unit, and execute step (3); Step (3) read the data state value Ds of each storage unit in sequence; if the data state value Ds flag is existence, write the key value Kv and the offset address offset of the management data to the allocation record table; if the data state value Ds flag is deletion, write the storage unit length len and the offset address offset of the management data to the recovery record table; thereby establishing the allocation record table and the recovery record table; Step (4) management software monitors user instruction: if the user instruction is query, execute steps (5)-(6); if the user instruction is delete, execute steps (7)-(8); if the user instruction is add, execute steps (9)-(11); Step (5) queries the allocation record table according to the key value Kv of the data required to be queried by the user; If the key value Kv exists in the allocation record table, read the data block of the cache area according to the offset address offset corresponding to the key value Kv, read the user data according to the user data length Dl recorded by the management data of the data block, and if the reading is successful, the query process is completed, and if the user data is not read in the cache area, execute step (6); If the key value Kv does not exist in the allocation record table, report that the data does not exist, and the query process is completed; Step (6) judges the state of the refresh value cache; If the refresh value cache is a refresh flag, erase the data of the corresponding storage unit according to the start and end addresses of the cache area, and write the cache area data into the corresponding storage unit; read the storage unit in the database file according to the offset address offset corresponding to the key value Kv, read the user data according to the user data length Dl recorded by the management data in the storage unit, and if the reading is successful, the query process is completed; If the refresh value cache is a non-refresh flag, read the storage unit in the database file according to the offset address offset corresponding to the key value Kv, read the user data according to the user data length Dl recorded by the management data in the storage unit, and if the reading is successful, the query process is completed; Step (7) queries the allocation record table according to the key value Kv of the data required to be deleted by the user; If the key value Kv exists in the allocation record table, read the data state value Ds of the management data of the data block of the cache area according to the offset address offset corresponding to the key value Kv; If the reading is successful, set the data state value Ds of the data block of the cache area to a deletion flag, set the refresh value cache to a refresh flag, and execute step (8); if the reading is unsuccessful, directly execute step (8); If the key value Kv does not exist in the allocation record table, report that the data does not exist, and the deletion process is completed; Step (8) judges the state of the refresh value cache; If the refresh value cache is a refresh flag, erase the data of the corresponding storage unit according to the start and end addresses of the cache area, and write the cache area data into the corresponding storage unit; then read the data of the cache area length in the database file into the cache area according to the offset address offset corresponding to the key value Kv, and set the data state value Ds of the corresponding data block to a deletion flag; write the offset address offset corresponding to the key value Kv and the storage unit length len corresponding to the offset address offset into the recovery record table, delete the key value Kv and the corresponding offset address offset in the allocation record table, and the deletion process is completed; If the flush value cache is the non-flush flag, read the data of the cache area length from the database file according to the offset corresponding to the key value Kv, set the data state value Ds of the corresponding data block to the deletion flag, and set the flush value cache to the flush flag; write the offset corresponding to the key value Kv and the storage unit length len corresponding to the offset into the recycle record table, delete the key value Kv and the corresponding offset in the allocation record table, and complete the deletion process; Step (9) finds the storage unit length len same as the length of the data to be added plus the length of the management data in the recycle record table; If the storage unit length len exists, find the data block corresponding to the offset in the cache area according to the offset corresponding to the storage unit length len, set the data state value Ds of the data block to the existence flag, write the data check value Dc and the key value Kv of the data to be added, erase the user data userdata of the data block, write the data to be added in the user data area, and set the flush value cache to the flush flag; execute step (10); If the storage unit length len does not exist, execute step (11); Step (10) writes the offset and the key value Kv into the allocation record table, deletes the offset and the corresponding storage unit length len in the recycle record table, and completes the addition process; Step (11) judges the state of the flush value cache; If the flush value cache is the non-flush flag, create a new data block in the cache area according to the maximum offset in the allocation record table, set the data state value Ds of the data block to the existence, write the data check value Dc, the user data length Dl and the key value Kv of the data to be added, and write the data to be added in the user data area; Set the flush value cache to the flush flag, write the offset and the key value Kv into the allocation record table, and complete the addition process; If the flush value cache is the flush flag, erase the data of the corresponding storage unit according to the start and end addresses of the cache area, and write the cache area data into the corresponding storage unit; create a new data block in the cache area according to the maximum offset in the allocation record table, set the data state value Ds of the data block to the existence, write the data check value Dc, the user data length Dl and the key value Kv of the data to be added, and write the data to be added in the user data area; write the offset and the key value Kv into the allocation record table, and complete the addition process.

3. The embedded lightweight database system of claim 1, wherein: The user data is the actual stored data, and the management data is the data created by the database for managing the user data, and the length of the management data is 9 bytes.

4. The embedded lightweight database system of claim 1, wherein: The length of the database identifier magic is 4 bytes.

Images