Database system and data management method
By introducing a data management unit into the database system, separating and managing data of database tables and managing data, the problem of long data input and output time in object storage is solved, and more efficient data management is achieved.
Patent Information
- Application Number
- JP2022037646
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2022-03-10
- Publication Date
- 2025-04-01
- Estimated Expiration
- 2042-03-10
AI Technical Summary
In database systems using object storage, the number and time of data input and output are long, and the prior art has failed to effectively reduce these times.
By introducing a data management unit into the database system, the data of the database table and the management data are divided into different object stores, and updated and managed separately, reducing the input and output operations of object stores.
It effectively reduces the time for data input and output and improves the performance of the database system.
Smart Images

Figure 0007654585000001 
Figure 0007654585000002 
Figure 0007654585000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a database system and a data management method. [Background technology]
[0002] In computer systems, databases (DB) are widely used as a method for managing data. A widely used method for databases is the relational database (RDB). In a relational database, data is managed in a tabular format, and the SQL language is used to access the data.
[0003] In recent years, relational databases (RDBs) that operate in cloud systems that share computer resources have become widespread. In cloud systems, data is stored in object storage. Object storage is a storage that divides data into objects and inputs and outputs the data. One characteristic of object storage is the long time it takes to input and output data. The time it takes to input and output data increases according to the number of objects that are read and written to the object storage.
[0004] A technology for applying object storage to a database system is described in Patent Document 1. Also, a study on accelerating data input / output is described in Patent Document 2. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] Patent No. 6730189 [Patent Document 2] Patent No. 4095840 Summary of the Invention [Problem to be solved by the invention]
[0006] In a database system that uses object storage, there is a demand for reducing the number of object input / output operations and shortening the time required for data input / output. However, the above technologies (Patent Documents 1 and 2) are not technologies that reduce the number of object input / output operations and shorten the time required for data input / output.
[0007] The present invention has been made to solve the above problems. That is, one of the objects of the present invention is to provide a database system and a data management method that can reduce the time required for inputting and outputting data. [Means for solving the problem]
[0008] In order to solve the above problems, the database system of the present invention is a database system in which a database including a table is constructed, and is equipped with a computer including a data management unit that manages the database, and an object storage, wherein the data management unit is configured to store in the object storage a first object that stores a first data area that includes data of the table of the database and a second data area that corresponds to the first data area and includes first management data that is updated in conjunction with updates to the data of the table, and to store in the object storage a second object that stores in the object storage a third data area that includes second management data related to the table, the second management data being not updated in conjunction with updates to the data of the table.
[0009] The data management method of the present invention is a data management method in a database system in which a database including a table is constructed, using a computer including a data management unit that manages the database, and an object storage, wherein the data management unit stores in the object storage a first object in which a first data area including data of the table of the database and a second data area that corresponds to the first data area and includes first management data that is updated in conjunction with updates to the data of the table, and stores in the object storage a second object in which a third data area is stored that includes second management data related to the table, the second management data being not updated in conjunction with updates to the data of the table.
[0010] The database system of the present invention comprises a computer including a data management unit that manages a database including a table, and a memory, and is connected to an object storage that stores the database. The data management unit updates and manages data of the database in an area of the memory, and stores the data from the memory to the object storage and reads the data from the object storage to the memory. The database data includes a first object in which a first data area that includes data of the table of the database and a second data area that corresponds to the first data area and includes first management data that is updated in conjunction with updates to the data of the table, and a second object in which a third data area that includes second management data related to the table and is not updated in conjunction with updates to the data of the table, is stored. The memory has a first cache area that performs LRU management to delete data based on access history, and a second cache area that does not perform the LRU management. The data management unit manages the first object in the first cache area and manages the second object in the second cache area. Effect of the Invention
[0011] According to the present invention, the time required for inputting and outputting data can be reduced. [Brief description of the drawings]
[0012] [Figure 1] FIG. 1 is a schematic diagram showing an example of the configuration of a database system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram for explaining an example of data stored in a first object. [Diagram 3] FIG. 3 is a diagram illustrating an example of data stored in the second object. [Figure 4] FIG. 4 is a diagram illustrating an example of the object management table. [Diagram 5] FIG. 5 is a diagram illustrating an example of the storage area management table. [Figure 6] FIG. 6 is a diagram for explaining an example of the import data. [Figure 7] FIG. 7 is a diagram illustrating an example of the import control information. [Figure 8] FIG. 8 is a flowchart showing the operational flow executed by the data import function of the main node. [Figure 9] FIG. 9 is a diagram illustrating an example of a search query received by the data search function. [Figure 10] FIG. 10 is a flow chart showing the operational flow executed by the data search function of the main node. [Figure 11] FIG. 11 is a flowchart showing the operation flow executed by the sub-node data search function of the sub-node. [Figure 12] FIG. 12 is a diagram for explaining an example of an update query received by the data updating function of the main node. [Figure 13] FIG. 13 is a diagram illustrating an example of an operational flow executed by the data updating function of the main node. [Figure 14] FIG. 14 is a diagram illustrating an example of an operational flow executed by the cache invalidation function of the main node. [Figure 15] FIG. 15 is a diagram illustrating an example of an operational flow executed by the sub-node cache invalidation function of the sub-node. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the following description, when describing identification information, expressions such as "ID" and "name" are used, but these are interchangeable. In addition, in the following description, a process may be described with a functional block as the subject, but the subject of the process may be a CPU instead of a functional block. Similarly, the subject of the process performed by executing a functional block may be a system, a computer, a node, etc. In addition, in the following description, various information may be described with expressions such as "table" and "record", but the various information may be expressed in a data structure other than these.
[0014] <<Embodiment>> 1, a database system according to an embodiment of the present invention includes a main node 101, a sub node 117, and an object storage 114. These are configured to be able to transmit and receive data (information) to and from each other via a network 113.
[0015] The main node 101 is a computer (server) and includes a memory 102a, a CPU 111a, and a storage device 112a. The computer may be a physical computer or a virtual computer. These are connected via a bus 110a. The main node 101 is connected to an external device (not shown) (e.g., an external terminal) so as to be able to transmit and receive data (information) to and from each other. The memory 102a includes a database 103a as software, and a first cache area 108a and a second cache area 109b as cache areas. The database 103a is software (database software) for realizing (constructing) a database in a database system, and is also referred to as a "data management unit" for convenience.
[0016] The database 103a includes a data import function 104, a data search function 105, a data update function 106, and a cache invalidation function 107. The functions realized by these will be described in detail later. The CPU 111 realizes the above-mentioned functions by executing a program contained in the database 103a stored in the memory 102a. The storage device 112a is a non-volatile storage device that can read and write data.
[0017] The secondary node 117 is a computer (server) and includes a memory 102b, a CPU 111b, and a storage device 112b. The computer may be a physical computer or a virtual computer. These are connected via a bus 110b. The memory 102b includes a database 103b as software, and a first cache area 108b and a second cache area 109b as cache areas. The database 103a is software (database software) for realizing (constructing) a database in a database system, and is also referred to as a "data management unit" for convenience.
[0018] The database 103b includes a sub-node data search function 118 and a sub-node cache invalidation function 119. The functions realized by these will be described in detail later. The CPU 111 realizes each of the above-mentioned functions by executing a program contained in the database 103b stored in the memory 102b. The storage device 112b is a non-volatile storage device from which data can be read and written.
[0019] The object storage 114 is a storage system that handles data in units of objects. An object includes data and metadata including an object ID. The object ID is identification information for identifying an object. The object storage 114 stores a first object 115 and a second object 116 as objects. The object storage 114 may be an object storage service such as "Amazon Simple Storage Service (Amazon S3) (registered trademark)". The first object 115 and the second object 116 will be described in detail later.
[0020] FIG. 2 is a diagram for explaining an example of data stored in a first object 115. In the object storage 114, database data is stored as an object. The database in this example is, for example, a relational database in a column store format, and data is managed in a tabular format. A table is made up of rows and columns. The horizontal direction of a table is called a row, and the vertical direction is called a column. The database includes a table in a tabular format and management data (first management data and second management data).
[0021] The database includes a first data area 202, a second data area 201, and a third data area 301 (see FIG. 3 described later), which are logical areas in which data is stored.
[0022] The first data area 202 and the second data area 201 are stored in the object storage 114 as a first object 115, and the third data area 301 is stored in the object storage 114 as a second object 116.
[0023] The first data area 202 is composed of units called pages 203. Data of the same column of a table is stored in a page 203. That is, some or all of the data of the same column of a table is stored in one page 203. The first data area 202 of the database is stored in one first object 115 if all columns of all rows of the table fit within the range of the maximum number of pages defined for one first object 115.
[0024] If the data of all columns of all rows of the table does not fit within the range of the maximum number of pages defined for one first object 115, the first data area 202 is divided into a plurality of small areas, and each of the divided small areas is stored in each of the plurality of first objects 115. Each of the plurality of small areas may also be referred to as a "divided first data area 202" or simply as a "first data area 202."
[0025] Specifically, if the data of all columns of all rows of a table does not fit within the range of the maximum number of pages defined for one first object 115, the first data area 202 is divided into multiple small areas in units of "multiple rows of the table (which may be a single row) and all columns of these rows." At this time, the first data area 202 is divided so that the divided small areas fit within the range of the maximum number of pages defined for one first object 115. Then, each of the divided multiple small areas (first data area 202) is stored in each of the multiple first objects 115.
[0026] The second data area 201 includes first management data corresponding to the first data area 202. The first management data includes, for example, an index, a row deletion flag, and the like. The index is, for example, a range index, and includes information indicating the storage page range of each column of the first data area 202, the value range of each column, and the like. The row deletion flag is a flag indicating that the corresponding row is considered to be a deleted row, and when the value is "1", it indicates that the corresponding row is considered to be a deleted row, and when the value is "0", it indicates that it is not. The first management data is data that is updated in conjunction with an update of table data, and when the data included in the corresponding first data area 202 is updated, it is updated so that the update contents are reflected.
[0027] When the first data area 202 is divided into a plurality of small areas, a second data area 201 is created containing first management data corresponding to each of the plurality of small areas (first data area 202). The small areas (divided first data area 202) and the second data area 201 containing the first management data corresponding to the small areas are stored in the first object 115. That is, in this case, one first object 115 stores one small area (divided first data area 202) and the second data area 201 corresponding to the small area.
[0028] Fig. 3 is a diagram for explaining an example of a third data area 301 stored in the second object 116. The third data area 301 includes second management data. As shown in Fig. 3, the second management data includes the object storage name of the object storage 114 in which the table is stored, the object storage path of the object storage, the maximum number of pages that can be stored in one object, the table name, and definition information of the table. The second management data is "data related to the table" that is not updated when the data of the table is updated.
[0029] 4 is a diagram for explaining an example of an object management table. The object management table TB1 is created when data is imported and stored in the storage device 112a. The object management table TB1 includes the object storage name of the object storage 114 in which the object is stored, the object storage path of the object storage 114, the table name of the table stored in the object storage 114, and the object ID of the object stored in the object storage 114.
[0030] 5 is a diagram for explaining an example of a storage area management table. The storage area management table TB2 is a table that stores management information of objects stored in the first cache area 108a, the second cache area 109a, and the storage device 112a of the main node 101, and the first cache area 108b, the second cache area 109b, and the storage device 112b of the secondary node 117. The storage area management table TB2 is stored in the storage devices 112a and 112b. The storage area management table TB2 includes a storage area name and an object ID of an object stored in the storage area corresponding to the storage area name.
[0031] The details of the above-mentioned data import function 104 will be described below with a specific example. Fig. 6 is a diagram for explaining an example of import data. As shown in Fig. 6, the import data 401 includes data (values) to be stored in a table of a database. Fig. 7 is a diagram for explaining an example of import control information. As shown in Fig. 7, the import control information 501 includes information indicating the object storage name of the object storage 114 that stores the import data, information indicating the object storage path, information indicating the maximum number of pages to be stored in one object (first object 115), information indicating the table name, and table definition information.
[0032] 8 is a flowchart showing an operation flow (processing flow) executed by the data import function 104. The following description is based on the assumption that the import data 401 does not fall within the range of the maximum number of pages defined for one first object 115. If the import data 401 falls within the range of the maximum number of pages defined for one first object 115, the process of step 803 is omitted, and a process that differs only in that "each small area" in step 804 is replaced with "first data area 202" is executed instead of step 804.
[0033] The data import function 104 starts processing from step 800 in FIG. 8, and executes the processing from step 801 to step 807 described below in order, and then proceeds to step 808 to temporarily end this operation flow.
[0034] Step 801 : The data import function 104 reads the input data including the import data 401 and the import control information 501 .
[0035] Step 802 : The data import function 104 generates data to be stored in the first data area 202 for each column of data in the import data 401 .
[0036] Step 803: The data import function 104 divides the first data area 202 of the database into small areas. Specifically, the data import function 104 determines how many rows of all-column data of the import data 401 should be put into one first object so that it fits into one "maximum number of pages that can be stored in a first object," and divides the first data area 202 of the database into small areas.
[0037] Step 804: The data import function 104 generates a second data area 201 for each small area.
[0038] Step 805: The data import function 104 compiles the first data area 202 and the corresponding second data area 201 into an object and stores it in the object storage 114. The first data area 202 and the second data area 201 are stored in one first object 115 for each divided small area.
[0039] Step 806 : The data import function 104 generates the third data area 301 of the database based on the import control information 501 .
[0040] Step 807: The data import function 104 stores the third data area 301 in the object. The data import function 104 creates an object management table TB1 and stores it in the storage device 112a.
[0041] The details of the above-mentioned data search function 105 will be described below with a specific example. Fig. 9 is a diagram for explaining an example of a search query received by the data search function 105. As shown in Fig. 9, a search query 901 indicates an SQL command for acquiring data from a table ("a command to acquire data with AGE between 30 and 60 from TABLE").
[0042] FIG. 10 is a flow chart showing the operation flow (processing flow) executed by the data search function 105 of the main node 101.
[0043] When the main node 101 receives the search query 901, the data search function 105 starts processing at step 1000 in FIG. 10 and proceeds to step 1001, where it refers to the storage area management table TB2 to determine whether the second object 116 is stored in the second cache area 109a.
[0044] If the second object 116 is not stored in the second cache area 109a, the data retrieval function 105 determines "No" in step 1001 and proceeds to step 1002 to store the second object 116 in the second cache area 109a. After that, the data retrieval function 105 proceeds to step 1003.
[0045] On the other hand, if the second object 116 is stored in the second cache area 109 a , the data retrieval function 105 determines “Yes” in step 1001 and proceeds directly to step 1003 .
[0046] When the data search function 105 proceeds to step 1003, it refers to the object management table TB1 to create a list of first objects 115 required to process the query (search query 901), and executes the processes of steps 1004 to 1009 described below in order, after which it proceeds to step 1010 and temporarily ends this operation flow.
[0047] Step 1004: The data search function 105 notifies the sub-node data search function 118 of the sub-node 117 of a part of the created list.
[0048] Step 1005: The data retrieval function 105 reads the first object 115 that has not been notified to the secondary node 117 into the storage area (storage device 112a).
[0049] Step 1006: The data search function 105 reads the page 203 required to execute the query from the first object 115 read into the storage area (storage device 112b) into the first cache area 108a. In the processes of steps 1005 and 1006, the data search function 105 performs LRU (Least Recently Used) management. According to the LRU management algorithm, when the capacity becomes full, the least accessed data is deleted and new data is added. While the first cache area 108a is managed by LRU (LRU control), no eviction occurs in the second cache area 109a, which has the effect of reducing the number of reads and writes to the second object 116.
[0050] Step 1007: The data retrieval function 105 executes the process of the query using the data read into the first cache area 108a.
[0051] Step 1008: The data retrieval function 105 receives the results of processing the query of the subnode 117.
[0052] Step 1009: The data search function 105 collects the query processing results of the main node 101 and the secondary node 117, and outputs the query processing results.
[0053] Fig. 11 is a flowchart showing an operation flow (processing flow) executed by the sub-node data search function 118 of the sub-node 117. The sub-node data search function 118 starts processing from step 1100 in Fig. 11, proceeds to step 1101, and refers to the storage area management table TB2 to determine whether or not the second object 116 is stored in the second cache area 109b.
[0054] If the second object 116 is not stored in the second cache area 109b, the sub node data search function 118 determines "No" in step 1101 and proceeds to step 1102 to store the second object 116 in the second cache area 109b. After that, the sub node data search function 118 proceeds to step 1105.
[0055] On the other hand, if the second object 116 is stored in the second cache area 109 b , the sub node data search function 118 determines “Yes” in step 1101 and proceeds directly to step 1105 .
[0056] When the sub-node data search function 118 proceeds to step 1105, it reads the first object 115 notified by the main node 101 into the storage area (storage device 112b).
[0057] Thereafter, the sub node data search function 118 executes the processes of steps 1106 to 1108 described below in order, and then proceeds to step 1109 to temporarily end this operational flow.
[0058] Step 1106: The secondary node data search function 118 reads the page 203 required to execute the query from the first object 115 read into the storage device 112b (storage area) into the first cache area 108b. In the processes of steps 1105 and 1106, the data search function 105 performs LRU management. According to the LRU management algorithm, when the capacity becomes full, the least accessed data is deleted and new data is added. While the first cache area 108b is managed by LRU (LRU control), no eviction occurs in the second cache area 109b, which has the effect of reducing the number of reads and writes to the second object 116.
[0059] Step 1107: The secondary node data search function 118 processes the query using the data read into the first cache area 108b.
[0060] Step 1108: The sub-node data search function 118 notifies the data search function 105 of the query processing result.
[0061] The details of the above-mentioned data update function 106, cache invalidation function 107, and sub-node cache invalidation function 119 will be explained below with concrete examples. Fig. 12 is a diagram for explaining an example of an update query received by the data update function. As shown in Fig. 12, an update query 1201 indicates an SQL command for updating table data ("a command to delete the row with ID=1 from TABLE").
[0062] FIG. 13 is a flowchart showing the operation flow (processing flow) executed by the data update function 106 of the main node 101.
[0063] When the main node 101 receives the update query 1201, the data update function 106 starts processing from step 1300 in FIG. 13 and executes the processing from step 1301 to step 1305 described below in order, and then proceeds to step 1306 to temporarily end this operational flow.
[0064] Step 1301: The data update function 106 uses the data search function 105 to identify the first object 115 to be updated in which the row to be updated is stored.
[0065] Step 1302 : The data update function 106 deletes the identified first object 115 from the object storage 114 .
[0066] Step 1303: The data updating function 106 updates the first object 115 and stores the new first object 115 in the object storage 114. If the update includes a DELETE process, the data updating function 106 changes the value of the row deletion flag in the second data area 201 corresponding to the relevant row of the first object 115 to be updated from "0" to "1" and creates a new first object 115.
[0067] Step 1304: The data update function 106 notifies the cache invalidation function 107 of information (for example, object ID) capable of identifying the object (first object 115) deleted from the object storage 114.
[0068] Step 1305: The data update function 106 receives a notification of the completion of processing (completion of invalidating the object) from the cache invalidation function 107.
[0069] FIG. 14 is a flowchart showing the operation flow (processing flow) executed by the cache invalidation function 107 of the main node.
[0070] The cache invalidation function 107 starts the process from step 1400 in FIG. 14, and executes the processes of steps 1401 and 1402 described below in order, and then proceeds to step 1403.
[0071] Step 1401: The cache invalidation function 107 receives, from the data update function 106, information capable of identifying the first object 115 deleted from the object storage 114 (for example, an object ID).
[0072] Step 1402: The cache invalidation function 107 notifies the secondary node cache invalidation function 119 of the received information capable of identifying the first object 115.
[0073] When the cache invalidation function 107 proceeds to step 1403, it refers to the storage area table TB2 to determine whether or not the object (that is, the object indicated by the information capable of identifying the first object 115) is stored in the storage device 112a.
[0074] If the object is stored in the storage device 112 a , the cache invalidation function 107 judges “Yes” in step 1403 and proceeds to step 1404 to delete the object from the storage device 112 a , and then proceeds to step 1405 .
[0075] On the other hand, if the object is not stored in the storage device 112 a , the cache invalidation function 107 judges “No” in step 1403 and proceeds directly to step 1405 .
[0076] When the cache invalidation function 107 proceeds to step 1405, it refers to the storage area table TB2 to determine whether or not the page 203 of the object is stored in the first cache area 108a.
[0077] If the page 203 of the object in question is stored in the first cache area 108a, the cache invalidation function 107 judges "Yes" in step 1405 and proceeds to step 1406, where it deletes the page 203 in question from the first cache area 108a, and then proceeds to step 1407.
[0078] On the other hand, if the page 203 of the object is not stored in the first cache area 108 a , the cache invalidation function 107 judges “No” in step 1405 and proceeds directly to step 1407 .
[0079] When the cache invalidation function 107 proceeds to step 1407, it waits for a response from the secondary node cache invalidation function 119. After that, the cache invalidation function 107 proceeds to step 1408 and temporarily ends this operation flow.
[0080] FIG. 15 is a flowchart showing the operation flow (processing flow) executed by the secondary node cache invalidation function 119.
[0081] The secondary node cache invalidation function 119 starts processing at step 1500 in FIG. 15, proceeds to step 1501 described below, and after receiving information capable of identifying the first object 115 to be invalidated, proceeds to step 1502.
[0082] When the secondary node cache invalidation function 119 proceeds to step 1502, it refers to the storage area table TB2 to determine whether the object in question (i.e., the object indicated by the information capable of identifying the first object 115 to be invalidated) is stored in the storage device 112b.
[0083] If the object is stored in storage device 112b, the secondary node cache invalidation function 119 judges "Yes" in step 1502 and proceeds to step 1503 to delete the object from storage device 112b, and then proceeds to step 1504.
[0084] If the object is not stored in the storage device 112 b , the secondary node cache invalidation function 119 determines “No” in step 1502 and proceeds directly to step 1504 .
[0085] In step 1504, the secondary node cache invalidation function 119 refers to the storage area table TB2 to determine whether or not the page 203 of the object is stored in the first cache area 108b.
[0086] If the page 203 of the object in question is stored in the first cache area 108b, the secondary node cache invalidation function 119 judges "Yes" in step 1504 and proceeds to step 1505, where it deletes the page 203 in question from the first cache area 108b, and then proceeds to step 1506.
[0087] On the other hand, if the page 203 of the object is not stored in the first cache area 108 b , the secondary node cache invalidation function 119 judges “No” in step 1504 and proceeds to step 1506 .
[0088] When the sub-node cache invalidation function 119 proceeds to step 1506, it notifies the completion of the processing to the cache invalidation function 107 of the main node 101. Thereafter, the sub-node cache invalidation function 119 proceeds to step 1507 and temporarily ends this operation flow.
[0089] <Effects> As described above, the database system using the object storage according to the embodiment of the present invention can solve the problem of data input / output taking a long time. In particular, the database system according to the embodiment of the present invention can shorten the data input / output time by reducing the number of object input / output operations performed in data search and data update, which are database operations.
[0090] <<Modifications>> The present invention is not limited to the above-described embodiment, and various modifications can be adopted within the scope of the present invention. For example, the format of the database in the present invention is not limited to the above-described example, and the database to which the present invention is applied may be a relational database in a row store format. [Explanation of symbols]
[0091] 101...main node, 102a, 102b...memory, 103a, 103b...database, 104...data import function, 105...data search function, 106...data update function, 107...cache invalidation function, 108a, 108b...first cache area, 109a, 109b...second cache area, 111a, 111b...CPU, 112a, 112b...storage device, 114...object storage, 115...first object, 116...second object, 117...secondary node, 118...secondary node data search function, 119...secondary node cache invalidation function
Claims
1. A database system in which a database including a table is constructed, a computer including a data management unit that manages the database; Object storage and Equipped with The data management unit a first data area containing data of the table of the database; a second data area corresponding to the first data area and including first management data that is updated in accordance with updates to the data in the table; storing a first object in the object storage; storing a second object in the object storage, the second object having a third data area including second management data related to the table, the second management data not being updated in association with an update of the data in the table; It was configured as follows: Database system.
2. 2. The database system according to claim 1, the computer comprises a memory including a first cache area and a second cache area; The data management unit storing data of the first object in the first cache area; storing data of the second object in the second cache area; It was configured as follows: Database system.
3. 3. The database system according to claim 2, The data management unit when a search query is received as an instruction to operate the database, determining whether data of the second object corresponding to the first object required for processing the search query is stored in the second cache area; if data of the second object corresponding to the first object necessary for processing the search query is not stored in the second cache area, reading the second object necessary for processing the search query from the object storage, and storing the read second object in the second cache area; It was configured as follows: Database system.
4. 3. The database system according to claim 2, The data management unit storing the first data area including all columns of all rows of the table in a tabular format in the first object; It was configured as follows: Database system.
5. 3. The database system according to claim 2, The data management unit storing, in each of the plurality of first objects, each of the plurality of first data areas divided into a plurality of small areas so as to include all columns of a portion of all rows of the table in tabular form; It was configured as follows: Database system.
6. 3. The database system according to claim 2, The computer includes a storage device, The data management unit storing the first object required for operating the database in the storage device; storing specific data of the first object stored in the storage device in the first cache area; It was configured as follows: Database system.
7. 7. The database system according to claim 6, The data management unit managing the specific data to be stored in the first cache area according to an LRU algorithm; It was configured as follows: Database system.
8. 7. The database system according to claim 6, The computer includes one main node and one or more sub-nodes, of the data management unit of the main node and the data management unit of the sub node, the data management unit of the main node is configured to store the first object and the second object in the object storage; When the data management unit of the main node receives a data update query as an instruction to operate the database, Executing a data update of the table of the database described in the data update query, creating a new first object in which the updated data is stored, and storing the new first object in the object storage; Invalidating the first object including the data to be updated that is stored in the storage device of the main node; Invalidating data of the first object including the data to be updated that is stored in the first cache area of the main node; the main node notifies the sub-node of information capable of identifying the first object including the data to be updated; The data management unit of the secondary node Invalidating the first object including the data to be updated that is stored in the storage device of the secondary node invalidating data of the first object to be updated that is stored in the first cache area of the secondary node; It was configured as follows: Database system.
9. 9. The database system according to claim 8, The data management unit of the main node is configured to determine whether or not the first object including the data to be updated is stored in the storage device of the main node, and when it is determined that the first object in which the data to be updated is stored is stored in the storage device of the main node, execute the invalidation to delete the first object including the data to be updated from the storage device of the main node; Furthermore, the data management unit of the main node determining whether or not the data of the first object including the data to be updated is stored in the first cache area of the main node, and if it is determined that the data of the first object including the data to be updated is stored in the first cache area of the main node, executing the invalidation to delete the data of the first object including the data to be updated from the first cache area of the main node; It was configured as follows: Database system.
10. 10. The database system according to claim 9, The data management unit of the secondary node is configured to determine whether or not the first object including the data to be updated and notified from the main node is stored in the storage device of the secondary node, and when it is determined that the first object in which the data to be updated is stored is stored in the storage device of the secondary node, execute the invalidation to delete the first object including the data to be updated from the storage device of the secondary node; Furthermore, the data management unit of the secondary node determining whether or not the data of the first object including the data to be updated is stored in the first cache area of the sub-node, and if it is determined that the data of the first object including the data to be updated is stored in the first cache area of the sub-node, executing the invalidation to delete the data of the first object including the data to be updated from the first cache area of the sub-node; It was configured as follows: Database system.
11. 7. The database system according to claim 6, The computers include a main node and one or more sub-nodes, of the data management unit of the main node and the data management unit of the sub node, the data management unit of the main node is configured to store the first object and the second object in the object storage; When the data management unit of the main node receives a search query as an instruction to operate the database, creating a list of the first objects required to process the search query and notifying the data manager of the secondary node of a portion of the list; storing the first object in the list that has not been notified to the data management unit of the secondary node in the storage device, reading data necessary for executing the search query from the first object stored in the storage device, and storing the data in the first cache area; performing processing of the search query using data stored in the first cache area; It was configured as follows: Database system.
12. 12. The database system according to claim 11, When the data management unit of the secondary node is notified of a part of the list from the main node, storing the first object of the list notified by the main node in the storage device, reading data necessary for executing the search query from the first object stored in the storage device, and storing the data in the first cache area; performing processing of the search query using data stored in the first cache area; It was configured as follows: Database system.
13. A data management method using a computer including a data management unit that manages a database in a database system including a table, and an object storage, comprising: The data management unit, a first data area containing data of the table of the database; a second data area corresponding to the first data area and including first management data that is updated in accordance with updates to the data in the table; storing a first object in the object storage; storing a second object in the object storage, the second object having a third data area including second management data related to the table, the second management data not being updated in association with an update of the data in the table; Data management methods.
14. A computer including a data management unit that manages a database including a table and a memory, In a database system connected to an object storage that stores the database, the data management unit updates and manages data of the database in an area of the memory, and stores data from the memory to the object storage and reads data from the object storage to the memory; The data in the database is a first object in which a first data area including data of the table of the database and a second data area including first management data corresponding to the first data area and updated in accordance with updates to the data of the table are stored; a second object in which a third data area is stored, the second management data being related to the table, the second management data being not updated in association with an update of the data of the table; Including, the memory has a first cache area that performs LRU management for deleting data based on an access history, and a second cache area that does not perform the LRU management; The data management unit manages the first object in the first cache area and manages the second object in the second cache area. A database system comprising:
Citation Information
Patent Citations
Database system
JP2003263350A
Information processing device, information processing system and method
JP2016212588A
Information processing apparatus and information processing program
JP2021076969A
Information processing program, information processing method, and information processing device
JP2022126099A
Cache memory management method
JP4095840B2