Storage control device and method

Abstract

To maintain both performance and cost in an environment in which the performance and cost differ on the basis of the number of times of writing from a storage control apparatus to an external storage and a database is stored in the external storage.SOLUTION: A database is stored in a plurality of pages that is a plurality of logical storage areas constituting a logical storage space. Data writing to the database in the logical storage space is performed in a page unit. A storage control apparatus coupled to an external storage and including a cache device writes data to the cache device in a page unit. The storage control apparatus determines whether an execution condition on external writing that is writing to the external storage is satisfied. When a result of the determination is true, the storage control apparatus writes data stored in the cache device to the external storage in a bulk unit that is larger than a page unit and that is a writing unit in which consistency is maintained.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates generally to storage control. [Background technology]

[0002] One method known for improving the I / O (Input / Output) performance of servers (server instances) in public clouds is to use a cache device built into the server. Because a cache device cannot store large amounts of data, data temporarily stored in the cache device is written to the server's external storage.

[0003] An example of data to be written is database data. Patent Document 1 discloses a technique for storing DB data in an HSM (Hierarchical Storage Management) system that uses a secondary storage (magnetic disk) and a tertiary storage (magnetic tape, etc.). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] US5,822,780 Summary of the Invention [Problem to be solved by the invention]

[0005] The server (an example of a storage control device) may be an on-premise server instead of or in addition to a server in a public cloud. An environment in which data is written from a cache device built into a server to an external storage of the server is an environment in which performance and cost differ based on the number of writes from the server to the external storage. Patent Document 1 does not disclose or suggest a technology for optimizing writing while maintaining data consistency in such an environment. [Means for solving the problem]

[0006] A database is stored in multiple pages, which are multiple logical storage areas that make up a logical storage space. The unit of writing data to the database in the logical storage space is a page. A storage control device that is connected to an external storage and has a cache device writes data to the cache device in page units. The storage control device determines whether or not the conditions for performing an external write, which is a write to the external storage, are met. If the result of the determination is true, the storage control device writes the data stored in the cache device to the external storage in batches, which are write units that are larger than a page unit and maintain consistency. Effect of the Invention

[0007] According to the present invention, it is possible to achieve both performance and cost in an environment in which performance and cost vary depending on the number of writes from the storage control device to the external storage and a database is stored in the external storage. [Brief description of the drawings]

[0008] [Figure 1] 1 shows the configuration of an entire system having a storage control system according to a first embodiment. [Diagram 2] This shows the relationship between the LBA space and the external storage space. [Diagram 3] 1 shows the configuration of an external management table. [Figure 4] 1 shows the configuration of a buffer management table. [Diagram 5] 1 shows the configuration of a cache management table. [Figure 6] 1 shows the configuration of an allocation rule table. [Figure 7] The flow of Insert / Import processing is shown below. [Figure 8] 13 shows the flow of a page allocation request process. [Figure 9] 1 shows the flow of area release processing. [Figure 10] The flow of Update / Delete processing is shown below. [Figure 11]The flow of the termination process is shown below. [Figure 12] 1 shows the configuration of an entire system having a storage control system according to a second embodiment. [Figure 13] 1 shows the configuration of an area management table. [Figure 14] The structure of the DB map table is shown below. [Figure 15] 13 shows the flow of a write request reception process. [Figure 16] 13 shows the configuration of an entire system having a storage control system according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] In the following description, an "interface device" may refer to one or more communication interface devices. The one or more communication interface devices may be one or more homogeneous communication interface devices (e.g., one or more NICs (Network Interface Cards)) or two or more heterogeneous communication interface devices (e.g., a NIC and an HBA (Host Bus Adapter)).

[0010] In the following description, a "memory" refers to one or more memory devices, which are an example of one or more storage devices, and may typically be a primary storage device. At least one memory device in the memory may be a volatile memory device or a non-volatile memory device.

[0011] In the following description, a "persistent storage device" may be one or more persistent storage devices, which are an example of one or more persistent storage devices. A persistent storage device may typically be a non-volatile storage device (e.g., an auxiliary storage device), and more specifically, may be, for example, a hard disk drive (HDD), a solid state drive (SSD), or a non-volatile memory express (NVMe) drive.

[0012] Furthermore, in the following description, a "processor" may be one or more processor devices. The at least one processor device may typically be a microprocessor device such as a CPU (Central Processing Unit), but may also be other types of processor devices such as a GPU (Graphics Processing Unit). The at least one processor device may be a single-core or multi-core. The at least one processor device may be a processor core. The at least one processor device may also be a broader processor device such as a hardware circuit (e.g., an FPGA (Field-Programmable Gate Array), a CPLD (Complex Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit)) that performs part or all of the processing.

[0013] In the following description, information that provides an output for an input may be described using expressions such as "xxx table," but the information may be data of any structure (for example, structured data or unstructured data), or may be a neural network that generates an output for an input, or a learning model such as a genetic algorithm or random forest. Therefore, the "xxx table" may be referred to as "xxx information." In the following description, the configuration of each table is an example, and one table may be divided into two or more tables, or all or part of two or more tables may be one table.

[0014] In the following description, functions may be described using the expression "yyy unit", but the functions may be realized by one or more computer programs being executed by a processor, or by one or more hardware circuits (e.g., FPGA or ASIC), or by a combination thereof. When a function is realized by a program being executed by a processor, the function may be at least a part of the processor, since the specified processing is performed using a storage device and / or an interface device, etc., as appropriate. Processing described with a function as the subject may be processing performed by a processor or a device having the processor. A program may be installed from a program source. The program source may be, for example, a program distribution computer or a storage medium (e.g., a non-transitory storage medium) that can be read by a computer. The description of each function is an example, and multiple functions may be combined into one function, or one function may be divided into multiple functions.

[0015] In addition, in the following description, when describing elements of the same type without distinguishing between them, a common reference symbol (e.g., 600) may be used, and when describing elements of the same type without distinguishing between them, reference symbols (e.g., 600a, 600b, 600c) may be used.

[0016] Several embodiments will be described below. In the following description, "DB" is an abbreviation for database. [First embodiment]

[0017] FIG. 1 shows the overall configuration of a system having a storage control system according to the first embodiment.

[0018] A client 400, an external storage 300, and a storage control system are connected to a communication network 50 (for example, the Internet). The storage control system is a computer system including a DBMS (DataBase Management System) 110.

[0019] The client 400 is an example of a sender of a DB operation request to the storage control system. A DB operation request may be called a DB query, and may be, for example, a Select, Update, Delete, Insert, or Import request.

[0020] The external storage 300 is a storage external to the storage control system. In this embodiment, the external storage 300 is a cloud storage (storage as a cloud computing service) in a public cloud.

[0021] The storage control system is a scale-out type computer system made up of multiple servers (nodes) 100. The server 100 is an example of a storage control device, and in this embodiment, is an on-premise server.

[0022] That is, in this embodiment, with regard to DB management, a configuration is adopted in which a plurality of on-premise servers constituting a storage control system share cloud storage.

[0023] The server 100 includes an interface device 106, a cache device 160, a memory 102 and a processor 104 connected thereto.

[0024] The interface device 106 communicates over a communications network 50 .

[0025] The cache device 160 may be a volatile storage device, but in this embodiment it is a persistent storage device (for example, an SSD (Solid State Drive) with an NVMe (Non-Volatile Memory Express) interface).

[0026] The memory 102 is, for example, a volatile memory (typically a main memory). The memory 102 stores software for implementing a DBMS (DataBase Management System) 110. The DBMS 110 is implemented by the processor 104 executing the software.

[0027] The DBMS 110 manages a DB buffer 112, an external management table 130, a buffer management table 114, a cache management table 124, and an allocation rule table 116. The DBMS 110 also has functions such as a DB processing unit 120 and an area management unit 122. These tables and functions will be described later.

[0028] The DB space (database address space) managed by DBMS 110 is made up of multiple pages. That is, for a database, the smallest unit of data is a page. In the following explanation, data in page units is called "DB data." Writing to cache device 160 is done in block units, which are smaller than the page size, but because the smallest writing unit by DBMS 110 is a page unit (DBMS 110 inputs and outputs data in page units), DB data is written to cache device 160. The page size may be an integer multiple of the block size.

[0029] In this way, data input / output to / from the DB buffer 112 and the cache device 160 is DB data 150, but the data unit input / output to / from the external storage 300 is an object 310. That is, the external storage 300 as cloud storage is an object storage in this embodiment. The object 310 is a collection of multiple DB data. That is, the size of the object 310 is larger than the size of the DB data (page). The size of the object 310 may be fixed, but is variable in this embodiment.

[0030] FIG. 2 shows the relationship between the LBA (Logical Block Address) space and the external storage space.

[0031] The LBA space is an example of a contiguous address space based on a block device inside the server 100. The LBA space is made up of a number of pages 600. Each of the multiple pages is classified into one of the segments. A "segment" is made up of two or more contiguous pages.

[0032] Each of the multiple segments is classified into one of the segment groups. A "segment group" is made up of two or more segments (for example, two or more consecutive segments).

[0033] Each segment is made up of multiple data pages 600c. The data pages 600c are pages in which the DB data 150 is stored (pages managed by the DBMS 110).

[0034] For each segment, a page 600 is used as a page management page 600b. The page management page 600b is a page in which page management information is stored. For each segment, the page management information includes information indicating the state of each data page 600c in the segment. The state of the data page 600c is, for example, "free" (the data page 600c is a free page) or "no free" (the data page 600c is not a free page). The page management table 600b may or may not be included in the segment (for example, the page management table 600b may be the first page in the segment).

[0035] Each segment group is composed of multiple segments. For each segment group, a page 600 is used as a segment management page 600a. The segment management page 600a is a page in which segment management information is stored. For each segment group, the segment management information includes information indicating the state of each segment in the segment group. The state of a segment may be, for example, "free" (there is a free page in the segment) or "no free" (there is no free page in the segment). In other words, the allocation of the page 600 to the cache device 160 is managed on a segment-by-segment basis. The segment management page 600a may or may not be included in any of the segments in the segment group (for example, the segment management page 600a may be the first page of the first segment of the segment group).

[0036] All data (including page management information) in multiple (or one) segments is written to the external storage 300 as one object 310. For this reason, the objects 310 include objects that include not only page management information but also segment management information, and objects 310 that include page management information but do not include segment management information as necessary. In this embodiment, one segment group is stored in the external storage 300 as one or more objects 310.

[0037] The number of data pages 600c in a segment may vary depending on the segment, but is fixed in this embodiment. Which page 600 becomes the segment management page 600a and which page 600 becomes the page management page 600b is determined according to the number of data pages 600c determined based on management information in the page 600a or 600b. Also, which object 310 contains the data in which page 600 may be determined by calculation from the page ID of the page 600. The correspondence between the page 600 and the object 310 is identified based on the allocation rule table 116 and the external management table 130. Also, the DB data may be a set of records in the DB. In addition, the data stored in data page 600c is accumulated in DB buffer 112, and the accumulated data is written to cache device 160 as DB data, but data as segment management information or page management information may be stored in an area of ​​DB buffer 112 or an area other than DB buffer 112 (an area in memory 102) and written from that area to cache device 160.

[0038] FIG. 3 shows the configuration of the external management table 130.

[0039] The external management table 130 is a table that indicates the relationship between elements in a DB and objects in the external storage 300. For ease of explanation, a DB table (a table in a DB) is taken as an example of the elements in the DB, but other elements such as an index may also be used.

[0040] The external management table 130 includes a DB management table 132 , an area management table 138 , and an unwritten page table 146 .

[0041] The DB management table 132 indicates the correspondence between DB tables and segments. The DB management table 132 has an entry for each DB table. Each entry has information such as a DB table ID 134 and a segment range 136.

[0042] The DB table ID 134 indicates the ID of the DB table. The segment range 136 indicates the range of one or more segments in which the DB table is stored (the ID of the first segment and the ID of the last segment).

[0043] The area management table 138 indicates the correspondence between a data insert / update process in a DB (for example, a process performed in response to a DB operation request such as Update, Delete, Insert, or Import) and a segment. The area management table 138 has an entry for each DB process. Each entry has information such as a process ID 140, a DB table ID 141, and a segment ID 144.

[0044] The process ID 140 indicates the ID of a DB process. The DB table ID 141 indicates the ID of a DB table to / from which data is input / output by the DB process. The segment ID 144 indicates the ID of a segment to / from which data is input / output by the DB process.

[0045] The unwritten page table 146 indicates the correspondence between DB processes and unwritten pages. An "unwritten page" is a page in which data that has not been written to the external storage 300 is stored. The unwritten page table 146 has an entry for each unwritten page. Each entry has information such as a process ID 143 and a free page ID 148.

[0046] The process ID 143 indicates the ID of the DB process. The unwritten page ID 148 indicates the ID of a page in which data has not been written to the external storage 300 among pages in which data has been inserted / updated by a data insert / update process in the DB.

[0047] The segment ID and page ID may be created by combining the device ID (typically a serial number) of a block device virtually assumed for ID generation with the first LBA of a segment or page. The ID of an object 310 (e.g., object name) and the storage location (offset) of that object 310 may be created from the values ​​of the segment ID and page ID. The ID of a segment including page 600 may be determined by calculation from the page ID of that page 600. On the other hand, the ID of a page 600 to be included in a segment may be determined by calculation from the ID of that segment.

[0048] FIG. 4 shows the configuration of the buffer management table 114.

[0049] The buffer management table 114 indicates the correspondence between buffer areas (areas in the DB buffer 112) and pages 600. The buffer management table 114 has an entry for each buffer area. Each entry has information such as a buffer area ID 172, a page ID 174, and a status 176.

[0050] The buffer area ID 172 indicates the ID of the buffer area. The ID of the buffer area may include the ID of the DB process for which data is input / output to the buffer area. The page ID 174 indicates the ID of the page 600 to which data is input / output to / from the buffer area. The status 176 indicates whether the data in the buffer area has been reflected (copied) to the cache device 160. "Clean" means that the data in the buffer area has been reflected to the cache device 160. "Dirty" means that the data in the buffer area has not been reflected to the cache device 160.

[0051] FIG. 5 shows the configuration of the cache management table 124.

[0052] The cache management table 124 indicates the correspondence between the LBA and the page 600. The cache management table 124 has an entry for each page 600. Each entry has information such as an LBA 178, a page ID 179, and a write status 182.

[0053] The LBA 178 indicates the first LBA of the page 600. The page ID 179 indicates the ID of the page 600. The write status 182 indicates whether all data in the page 600 has been written to the external storage 300 or not.

[0054] FIG. 6 shows the configuration of the allocation rule table 116 .

[0055] The allocation rule table 116 represents the relationship between pages, segments, segment groups, and objects 310. The allocation rule table 116 includes information such as a page size 184, a segment size 186, a segment group size 188, and an object base size 192.

[0056] The page size 184 represents the size of the page 600. The segment size 186 represents the size of the segment (specifically, the number of pages 600 that make up the segment). The segment group size 188 represents the size of the segment group (specifically, the number of segments that make up the segment group). The object base size 192 represents the base size of the object 310.

[0057] In this embodiment, the basic size of the object 310 is the size of the object 310 as a set of segments that do not have a segment management page 600a. According to the example shown in Fig. 6, the basic size of the object 310 is the size of four segments that do not have a segment management page 600a. For example, if the object 310 includes one segment that has a segment management page 600a, the size of the object 310 is the basic size of the object 310 plus the size of the segment management page 600a. In this case, the page ID of the segment management page 600a and the object 310 that includes it may be calculated based on the information stored in the allocation rule table 116 according to the rule illustrated in Fig. 2.

[0058] An example of the processing performed in this embodiment will now be described.

[0059] Figure 7 shows the flow of the Insert / Import process.

[0060] The Insert / Import process is a process that is performed by the DBMS 110 in response to an Insert or Import request received by the DBMS 110 as a DB operation request.

[0061] The area management unit 122 performs an initial page allocation process (S1001). The initial page allocation process is a process of allocating the first data page 600c into which the data to be inserted or imported is written. The "first data page 600c" is the data page 600c identified from the allocation rule table 116. For example, multiple page IDs of the segment management information are calculated based on the allocation rule table 116, and a segment having a free page is found by referring to them. The page management information of the found segment is referenced, and the data page 600c is identified from the page ID having a free page. At this time, the page ID of the segment management information of the segment to which the most recent page allocation was performed may be saved, and used preferentially.

[0062] If the page allocation in S1001 fails (S1002: Failed), the process ends with an error. If there is no corresponding page in S1001 (S1002: Not Corresponding), a page allocation request process is performed (S1012).

[0063] If the corresponding page is found in S1001 (S1002: Success), the DB processing unit 120 inserts or imports data for one record into the buffer area corresponding to the allocated page (corresponding page) (S1003). If the process of S1003 fails (S1004: Failure), the process ends with an error.

[0064] If there is no free space for one record in the buffer area (allocated page) in S1003 (S1004: no free space), the DB processing unit 120 writes all data (DB data) in the buffer area to the allocated page based on the cache device 160 (S1011). After that, a page allocation request process is performed (S1012).

[0065] If there is no free space for one record in the buffer area (allocated page) in S1003 (S1004: no free space), the DB processing unit 120 judges whether the next record data exists or not (S1005). For example, if the DB operation request is an Insert request and the destination of the next data to be inserted is the same DB table, the judgment result in S1005 may be true (exists). On the other hand, if the destination of the next data to be inserted is a different DB table or if the DB operation request is not an Insert request, the judgment result in S1005 may be false (does not exist).

[0066] If the determination result in S1005 is true (S1005: exists), the process returns to S1003.

[0067] If the determination result of S1005 is false (S1005: not present), the DB processing unit 120 writes all data (DB data) in the buffer area to the assigned page based on the cache device 160 (S1006), except for exceptional cases (specifically, a case where the DB operation request is an Insert request and Commit has not been issued). After that, the process ends normally.

[0068] When a page is allocated in S1001 or S1012, the area management unit 122 secures a buffer area corresponding to the page allocation from the DB buffer 112 (adds an entry corresponding to the buffer area to the buffer management table 114).

[0069] Furthermore, in S1011 and S1006, the following may be adopted. The area management unit 122 updates the status 176 (status 176 in the buffer management table 114) corresponding to the buffer area in which the data to be cached is stored to “clean”, and also registers the ID of the assigned page in the unwritten page table 146 as the unwritten page ID 148 corresponding to the process ID of this Insert / Import process. The allocated page may be a data page whose page ID 179 in the cache management table 124 is an invalid value, or a data page whose write status 182 in the cache management table 124 is "done."

[0070] FIG. 8 shows the flow of the page allocation request process (S1012 in FIG. 7).

[0071] The area management unit 122 searches for a free page to be allocated (S1101). Specifically, for example, the following may be performed. (S1101-1) Region management unit 122 identifies a segment including a previously allocated page from segment ID 144 in region management table 138. Region management unit 122 references page management information in the identified segment and searches for a free page in the segment. (S1101-2) If no free page is found in S1101-1, the area management unit 122 refers to segment management information in the segment group including the segment identified in S1101-1, and searches for a segment having a free page, for example, in one direction. Here, "searching in one direction" means searching in the direction in which the segment ID value increases, and returning to the first segment ID (segment ID with the minimum value) if no free page is found even when the segment ID value reaches the maximum value. (S1101-3) If a segment having a free page is found in S1101-2, the area management unit 122 sets the free page in the found segment as an assigned page.

[0072] If no free page is found in S1101 (S1102: No), the process ends with an error.

[0073] In S1101, if a free page is found (S1102: Yes), the area management unit 122 judges whether the data of the found free page is included in the same object 310 as the data of the previously allocated page (S1103). This judgment may be, for example, a judgment of whether the ID of an object specified by calculation using the page ID of the previously allocated page matches the ID of an object specified by calculation using the page ID of the found free page.

[0074] If the determination result in S1103 is true (S1103: Yes), the process ends normally.

[0075] If the determination result of S1103 is false (S1103: No), external writing is performed (S1104). That is, the area management unit 122 writes the object 310 including the data of all pages corresponding to the all unwritten page ID 148 in which the same object is specified, to the external storage 300. Thereafter, the process ends normally. Details of S1104 are as follows, for example. Data in a page that is not registered as an unwritten page ID 148 may be data in an existing object 310. For example, if a newly written object 310 contains a mixture of data in a page that is registered as an unwritten page ID 148 and data in a page that is not registered as an unwritten page ID 148, the latter data may be data obtained from the existing object 310. If the buffer management table 114 contains an entry that includes a page ID 174 that matches the unwritten page ID 148 and a status of "dirty", the data in the buffer area may be included in the object 310. In this case, the page in the "dirty" status may be written to the cache device 160 at the same time. If the buffer management table 114 contains an entry that includes a page ID 174 matching the unwritten page ID 148 and a state of “clean”, the area management unit 122 may identify a page ID 179 matching the page ID 174 from the cache management table 124. The data in this buffer area may be included in the object 310. After writing of the object 310 is completed, if the writing status 182 corresponding to the page ID 179 is "not done", the area management unit 122 may update the writing status 182 to "done". The area management unit 122 may update the page management information, and write the object 310 including the updated page management information to the external storage 300. The area management unit 122 may also update the segment management information, and write the object 310 including the updated segment management information to the external storage 300.

[0076] FIG. 9 shows the flow of area release processing.

[0077] For example, the area release process may be performed in response to a request to delete a DB table, or may be performed when a request to release area is received.

[0078] The area management unit 122 identifies the page IDs of all pages as the released area (S1201). For example, the page IDs of all pages as the released area are identified based on the segment range 136 corresponding to the DB table to be deleted. Also, when an area release request is received, the data page 600c included in the specified range may be referenced to check whether all data in the page satisfies the conditions of the area release request, and if the conditions are satisfied, the page ID may be identified as the page to be released.

[0079] The area management unit 122 invalidates (S1202) any entry in the buffer management table 114 that includes a page ID 174 that matches the page ID identified in S1201.

[0080] The area management unit 122 invalidates (S1203) any entry in the cache management table 124 that includes a page ID 174 that matches the page ID identified in S1201.

[0081] The area management unit 122 updates the information and writes externally (S1204). Specifically, the area management unit 122 updates the page management information in the segment including the page specified in S1201 to information indicating that the page is an empty page. Also, the area management unit 122 updates the segment management information in the segment group including the page specified in S1201 to information indicating that the segment has the empty page. The area management unit 122 writes the object 310 including the segment whose page management information has been updated and the object 310 including the segment whose segment management information has been updated to the external storage 300. These objects 310 may be the same object.

[0082] In the area release process, so-called read-modify-write may be performed in units of objects 310 or in units of pages. That is, page management information to be updated (segment management information to be updated) in the segment (segment group) including the page identified in S1201 may be read from the external storage 300 by the DB processing unit 120, the management information may be updated by the area management unit 122, and the updated management information may be included in the corresponding object 310 in the external storage 300 by the DB processing unit 120. In this way, the page identified in S1201 is made free by updating the page management information and segment management information.

[0083] FIG. 10 shows the flow of the Update / Delete process.

[0084] The update / delete process is a process that is performed by the DBMS 110 in response to an update or delete request received by the DBMS 110 as a DB operation request.

[0085] The DB processing unit 120 updates or deletes data in a buffer area corresponding to the Update / Delete process (S1301). For example, if there is an entry corresponding to the buffer area in the cache management table 124, the DB processing unit 120 may read DB data from a page corresponding to the entry into the buffer area and update or delete the data in the buffer area. Also, for example, if there is no entry corresponding to the buffer area in the cache management table 124, the DB processing unit 120 may read DB data from an object 310 (object 310 in the external storage 300) including DB data of a page corresponding to the buffer area into the buffer area and update or delete the data in the buffer area.

[0086] The area management unit 122 updates the unwritten page table as necessary (S1302). For example, if the buffer area in which the data has been updated or deleted in S1301 is a newly allocated buffer area, an entry including the ID of the page corresponding to the buffer area as the unwritten page ID 146 may be added to the unwritten page table 146.

[0087] FIG. 11 shows the flow of the termination process.

[0088] The termination process is performed when a Commit is issued or when an Import process ends.

[0089] The area management unit 122 identifies an entry having the process ID of a completed process (for example, an entry including a segment ID 144 or an unwritten page ID 148) from the external management table 130 (S1401).

[0090] The area management unit 122 identifies the object 310 in which the data of the page is stored, based on the ID of the page indicated by the entry identified in S1401 (S1402).

[0091] The area management unit 122 writes the data of the page indicated by the entry identified in S1401 and management information related to the page (page management information and segment management information) to the external storage 300 (S1403). The writing in S1403 is performed in object 310 units.

[0092] In this embodiment, the following may be adopted for writing to the external storage 300. In a write request to the external storage 300 on an object-by-object basis, the object to be updated (written) may be associated with an object ID (object name). This causes the external storage 300 to perform writing in accordance with the request (writing the entire object 310). In a write request to external storage 300 on an object-by-object basis, the data to be updated, the object ID (object name), and the offset from the beginning of object 310 may be associated with each piece of data (DB data, page management information, or segment management information) to be updated (written) in object 310. This allows external storage 300 to perform writing in accordance with the request (partial update of object 310 (update in data units)). Note that if partial writing of object 310 is not permitted, the object 310 is read, the data to be updated is replaced, and the data is written back.

[0093] In addition, in this embodiment, the following may be adopted for reading from the external storage 300. A read request to the external storage 300 for each object may be associated with the object ID (object name) of the object to be read. This causes the external storage 300 to read in accordance with the request (write the entire object 310). In a read request to the external storage 300 on an object-by-object basis, an object ID (object name) and an offset from the beginning of the object 310 may be associated with each data to be read in the object 310. This allows the external storage 300 to read in accordance with the request (partial reading from the object 310 (reading on a data-by-data basis)).

[0094] The above is the description of the first embodiment. When the DBMS 110 of any of the servers 100 receives a DB operation request specifying a DB table, the DBMS 110 may notify the other servers 100 of the DB table ID of the specified DB table. When the DBMS 110 of any of the servers 100 performs external writing (when an object 310 is newly written or when any object 310 in the external storage 300 is updated), the DBMS 110 may notify the other servers 100 of the object ID of the object 310 that has been written (newly written or updated). [Second embodiment]

[0095] The second embodiment will be described below, focusing mainly on the differences from the first embodiment, and explanations of the points in common with the first embodiment will be omitted or simplified.

[0096] FIG. 12 shows the overall configuration of a system having a storage control system according to the second embodiment.

[0097] The DBMS 110 does not need to have a cache management table 124. Moreover, the external management table 130 does not need to have an unwritten page table 146. Therefore, S1302 in FIG. 10 does not need to be included. Moreover, the termination process in FIG. 11 may be performed on all of the data in the DB table that is the processing target. The area to be processed may be identified based on the DB management table 132 in the external management table 130.

[0098] Each server 100 has a software defined storage (SDS) 210 in addition to the DBMS 110. Specifically, the SDS 210 is realized in the server 100 by the processor 104 executing software for realizing the SDS 210.

[0099] The SDS 210 manages a DB map table 250 and a storage management table 240. The SDS 210 also includes a storage management unit 220.

[0100] The storage manager 220 of the SDS 210 provides a logical device to the DBMS 110. A logical device is a logical storage device, for example, a logical cache device based on the cache device 160. The storage space of the logical device may be a contiguous LBA space as illustrated in FIG.

[0101] The DB processing unit 120 of the DBMS 110 inputs and outputs data to and from the logical device provided by the SDS 210. The memory management unit 220 of the SDS 210 inputs and outputs data input and output to and from the logical device to the cache device 160 (the memory management unit 220 of the SDS 210 inputs and outputs data to and from the external storage 300). For example, when data is written from the DBMS 110 to the logical device, the memory management unit 220 of the SDS 210 first writes the data to the cache device 160. Later, when the area management unit 122 of the DBMS 110 instructs the memory management unit 220 of the SDS 210 to write to the external storage 300, the memory management unit 220 of the SDS 210 writes the data in the cache device 160 to the external storage 300. More specifically, the following is adopted. 7, the DBMS 110 writes data to the logical device. Prior to this writing, data is also written to the logical device as related management information (page management information or segment management information related to the page to be updated). In S1104 in FIG. 8, S1204 in FIG. 9, and S1403 in FIG. 11, the DBMS 110 instructs the SDS 210 to write to the external storage 300, and the SDS 210 writes to the external storage 300 in response to the instruction.

[0102] The SDS 210 does not need to hold the allocation rule table 116, which is information related to the area mapping of the DBMS 110. In this case, the storage management table 240 holds entries without breaking down the write unit from the DBMS 110 to the logical device. In other words, mapping is performed without dividing the write unit from the DBMS 110, and writing to the external storage 300 is performed without dividing the write unit from the DBMS 110.

[0103] The DBMS 110 performs mutual conversion between page IDs and segment IDs, and logical device IDs and logical device internal addresses (LBA).

[0104] FIG. 13 shows the configuration of the storage management table 240.

[0105] The storage management table 240 indicates the relationship between the address of the logical device, the address of the cache device 160, and the storage location in the external storage 300. The storage management table 240 has an entry for each unit of writing from the DBMS 110 to the logical device. Each entry has information such as a logical address 242, a cache address 244, an external storage location 246, and a status 248.

[0106] The logical address 242 indicates the address range of a logical device. The logical address 242 is expressed, for example, as a combination of the ID of the logical device and the range of the LBA.

[0107] The cache address 244 indicates the address range of the cache device 160. The cache address 244 is expressed, for example, as a combination of the ID of the cache device 160 and the range of LBAs.

[0108] The external storage location 246 indicates a storage location in the external storage 300. The external storage device 246 is expressed, for example, by a combination of an object ID (object name) and an offset from the beginning of the object.

[0109] The status 248 indicates in which storage device the data written in the write unit exists. "Both" means that the data exists in both the cache device 160 and the external storage 300. "Str only" means that the data does not exist in the cache device 160 but exists in the external storage 300.

[0110] FIG. 14 shows the configuration of the DB map table 250.

[0111] The DB map table 250 shows the correspondence between DB tables, segments, and logical device addresses. The DB map table 250 has an entry for each DB table. Each entry has information such as a DB table ID 334, a segment range 336, and a logical address 342.

[0112] The DB table ID 334 indicates the ID of the DB table. The segment range 336 indicates the range of one or more segments in which the DB table is stored (the ID of the first segment and the ID of the last segment). The logical address 342 indicates the address range of the logical device.

[0113] FIG. 15 shows the flow of the write request acceptance process.

[0114] The write request reception process is a process performed by the storage management unit 220 of the SDS 210. The write request reception process is a process performed in response to an external write request from the DBMS 110. The external write request is issued by the DBMS 110 in S1403 of the termination process by the DBMS 110 (see FIG. 11). That is, in the second embodiment, in S1403, an external write request is issued instead of performing an external write.

[0115] The storage management unit 220 writes data to the cache device 160 (S1501). The data written here is data specified by the DBMS 110 (DB data and related management information).

[0116] The storage management unit 220 updates the storage management table 240 (S1502). Specifically, the storage management unit 220 adds an entry corresponding to the unit of writing in S1501 to the storage management table 240. The value of the status 248 in the added entry may be a value that means that the data exists in the cache device 160 but does not exist in the external storage 300.

[0117] The memory management unit 220 judges whether or not external writing of the previously written data (data written in S1501 of the previous write request reception process) is necessary (S1503). Specifically, for example, the memory management unit 220 identifies an entry that indicates a logical address within a certain distance in the direction of smaller addresses from the logical address indicated by the entry added in S1502. The memory management unit 220 judges whether or not there is an entry having a status 248 indicating that there is data in the cache device 160 but no data in the external storage 300 among the identified entries. If this judgment result is true, the judgment result of S1503 is true (i.e., judgment that external writing is necessary), and if this judgment result is false, the judgment result of S1503 is false (i.e., judgment that external writing is not necessary). If the judgment result of S1503 is false (S1503: No), the process ends normally without performing S1504.

[0118] If the determination result of S1503 is true (S1503: Yes), the storage manager 220 performs external writing of the data at the location indicated by the cache address 244 of the entry to be written (entry in the storage management table 240) (S1504). This external writing is performed according to the write unit indicated by the entry.

[0119] The above is a description of the second embodiment. [Third embodiment]

[0120] The third embodiment will be described below, focusing mainly on the differences from the first and second embodiments, and explanations of the commonalities between the first and second embodiments will be omitted or simplified.

[0121] FIG. 16 shows the overall configuration of a system having a storage control system according to the third embodiment.

[0122] The SDS 210 manages an external write rule table 230 that indicates the timing of external write. The storage management unit 220 of the SDS 210 determines the timing of external write based on the external write rule table 230.

[0123] When the DBMS 110 performs the termination process (see FIG. 11), the storage manager 220 receives an external write request from the DBMS 110, and in response to the request, performs the process shown in FIG.

[0124] This concludes the description of the third embodiment.

[0125] The following is a summary of the first to third embodiments. The summary may include supplementary explanations to the above explanations, or may include explanations of modified examples.

[0126] The server 100 is an example of a storage control device. The server 100 is connected to an external storage 300, includes a processor 104 and a cache device 160, and updates a database.

[0127] The database is stored in multiple pages 600 (multiple logical storage areas) that make up the logical storage space. The unit of data writing to the database in the logical storage space is the page unit. The processor 104 writes data to the cache device 160 in page units. The processor 104 determines whether or not the conditions for performing an external write (writing to the external storage 300) are met. If the result of the determination is true, the processor 104 writes the data stored in the cache device 160 to the external storage 300 in object units. The object unit is an example of a bulk unit that is a write unit that is larger than the page unit and maintains consistency, and the data in the object unit is the object 310.

[0128] In this manner, the database data is stored in the external storage 300. In an environment in which the performance and cost vary depending on the number of writes from the server 100 to the external storage 300 and the database is stored in the external storage 300, low-overhead writing while maintaining consistency, that is, write optimization that achieves both performance and cost, is expected.

[0129] The processor 104 may write page management information for each segment including multiple data pages 600c. For each segment, the data page 600c is a page in which DB data is stored, and the page management information is management information indicating which data page of the segment is free. In the above-mentioned embodiment, the page management information may be written to a page management page 600b secured for each segment from multiple pages 600. The processor 104 includes page management information for the segment including the data page in addition to the DB data in the data page 600c in the object 310 written to the external storage 300. In one object 310, the page management information may include the page ID of the data page 600c for each data page 600c in the segment corresponding to the page management information, and may also include, for each data page 600c, the position of the DB data of the data page 600c (offset from the object 310) or information indicating whether the data page 600c is free. In one object 310, the DB data of all data pages 600c in the segment corresponding to the page management information may be included, or the page management information may include DB data of some of the data pages 600c in the segment corresponding to the page management information, with the page management information indicating that some of the data pages 600c are free.

[0130] The logical storage space in which the database is stored is a continuous address space, but the data in the continuous address space must be divided into multiple objects 310 and stored in the external storage 300. By storing page management information in addition to DB data in the object 310, it is possible to identify the DB data based on the page management information in the object 310. In addition, since the page management information and DB data are stored in the external storage 300 as the object 310 while maintaining consistency, low overhead writing while maintaining consistency is expected. Segments may be of fixed size or variable size (segments may be of uniform or non-uniform size).

[0131] The processor 104 may write segment management information for each segment group including a plurality of segments. For each segment group, the segment management information may be management information indicating which segment of the segment group has an empty data page. The processor 104 may include, in the object 310 written to the external storage 300, the DB data in the data page 600c and the page management information for the segment including the data page 600c, as well as segment management information for the segment of the segment group including the segment if the segment management information is associated with the segment. In one object 310, the segment management information may include a segment ID for each segment in the segment group corresponding to the segment management information. The segment ID may be calculated from the page ID, and the ID of the object 310 may be calculated from the page ID or the segment ID.

[0132] A segment with a free page is identified from the segment management information, and the free page can be identified from the page management information of the identified segment. This makes it possible to quickly identify a free page. Note that a "free page" is a page to which new data may be written.

[0133] In the above-described embodiment, the segment management information may be written to a segment management page 600a secured for each segment group from multiple pages 600. In other words, a segment group may be configured with the maximum number of segments that can be managed by information that can be stored in one page 600. This makes it possible to quickly identify free pages without changing the writing unit, which is a page unit.

[0134] The condition for performing the external write may be that any of the data pages 600c is released. When any of the data pages 600c is released, the processor 104 may update the page management information for the segment including the released data page 600c, and write the object 310 including the updated page management information to the external storage 300. In this object, DB data does not exist for the released data page 600c, and the updated page management information manages the data page 600c as a free page. In this way, at the timing of the release of the data page 600c, the page management information of the object 310 is updated, and external write (update) of the object 310 including the updated page management information can be performed.

[0135] The processor 104 may be configured to perform a write process (e.g., an insert / import process) that is a process of writing data to a database in response to a write request. In the write process, the processor 104 may determine whether the object 310 to which the data page 600c to which the data is to be written belongs is a data page 600c different from the data page 600c, and is the same as the object 310 to which the data page 600c to which the data was previously written belongs. The condition for performing the external write may be that the result of the determination is false.

[0136] In a write process, when data is written to two or more data pages 600c at consecutive addresses, when the write destination shifts from the first data page 600c to the next second data page 600c, the object 310 to which the data page 600c including the first data page 600c belongs may be different from the object 310 to which the second data page 600c belongs. In this case, the object 310 to which the data page 600c including the first data page 600c belongs is externally written. In this way, low overhead writing while maintaining consistency is expected.

[0137] The condition for performing the external write may be that the above-mentioned write process is completed. When the write process is completed, the processor 104 may identify an unwritten data page 600c that corresponds to the write process and is an unwritten data page 600c that corresponds to data that has been written to the cache device 160 but not written to the external storage 300. The processor 104 may write an object 310 that corresponds to the identified unwritten data page 600c and includes data that exists in the cache device 160 to the external storage 300. In this way, even if unwritten data (data that has not been written to the external storage 300) equivalent to the size of the object 310 is generated, low overhead writing while maintaining consistency is expected.

[0138] The processor 104 may execute a first software for making the processor 104 function as the DBMS 110 for writing data to a database, and a second software for making the processor 104 function as the SDS 210 for writing data to the cache device 160 and the external storage 300. The SDS 210 may provide the DBMS 110 with a logical device, which is a logical storage space. The SDS 210 may accept a page-by-page write request to the logical device from the DBMS 110, and write data to the cache device 160 in response to the write request. The SDS 210 may determine whether or not an external write execution condition is satisfied. If the result of the determination is true, the SDS 210 may write data stored in the cache device 160 to the external storage 300 in object units. This is expected to result in low overhead writing with little modification of the DBMS 110 and maintaining consistency.

[0139] The condition for performing external writing may be that a logical address that satisfies the following conditions exists. A logical address that is within a certain distance from the current destination logical address in the direction of smaller addresses. A logical address corresponding to data that exists in the cache device 160 but does not exist in the external storage 300.

[0140] The object 310 may be data including data of multiple DBMS write units. The DBMS 110 write unit may be a unit of writing data from the DBMS 110 to a logical device. In this way, the object 310 including data of multiple DBMS write units is written externally while maintaining the DBMS write unit (the unit of writing from the DBMS 110 to a logical device). For this reason, low overhead writing while maintaining consistency is expected.

[0141] The conditions for performing external writing may be conditions represented by information provided from the DBMS 110 to the SDS 210, or conditions represented by information predefined in the SDS 210. For example, the external writing rule table 230 (see FIG. 16) managed by the SDS 210 may be a table in which information received from the DBMS 110 is stored, or may be a predefined table. This allows the SDS 210 to perform external writing at an appropriate timing, similar to the DBMS 110 in the first embodiment.

[0142] The server 100 is an on-premise device (physical computer), the cache device 160 is a block device, and the external storage 300 is an object storage. Since block access may be maintained for data writing within the server 100 (data may be written to the cache device 160 in page units by block access), it is expected that write optimization that balances performance and cost with little modification of the server 100 can be realized.

[0143] Although several embodiments have been described above, these are merely examples for explaining the present invention, and the scope of the present invention is not limited to these embodiments. The present invention can be implemented in various other forms. For example, the server 100 may be a server in a cloud such as a public cloud (a server instance as a cloud computing service) instead of an on-premise server. The storage control system may be configured with one server 100. Although the external storage 300 is an object storage, it may be a file storage and the object 310 may be a file. A file system may be constructed on the cache device 160, DB data 150 may be stored in a file, and a file name and offset may be used in place of the cache LBA 178 in the cache management table 124. [Explanation of symbols]

[0144] 100...Server, 300...External storage

Claims

1. A storage control device that is connected to an external storage device, includes a processor and a cache device, and updates a database, The database is stored in a plurality of pages which are a plurality of logical storage areas constituting a logical storage space; The unit of data writing to the database in the logical storage space is a page unit, The processor, writing data to said cache device in pages; determining whether or not a condition for performing external writing, which is writing to the external storage, is satisfied; If the result of the determination is true, writing the data stored in the cache device to the external storage in a batch unit that is a write unit larger than the page unit; The external storage and the collective unit are either of the following (A) and (B), (A) the external storage is an object storage, and the collective unit is an object unit in which all data in one or more segments is treated as an object. (B) the external storage is a file storage, and the collective unit is a file unit in which all data in one or more segments is treated as a file; the processor is adapted to write page management information for each segment including a plurality of data pages; For each segment: A data page is a page where database data is stored. The page management information is management information that indicates which data pages in the segment are free. the processor causes a batch of data to be written to the external storage to include, in addition to the data in a data page, page management information for a segment including the data page; Storage control device.

2. The external storage and the collective unit are (A), The processor is configured to write segment management information for each segment group including a plurality of segments, For each segment group, the segment management information is management information indicating which segment of the segment group has a free data page, the processor includes, in the batch of data written to the external storage, not only the data in the data page and page management information for the segment including the data page, but also segment management information for a segment group including the segment if segment management information is associated with the segment; The storage control device according to claim 1.

3. the execution condition is that any data page is released, when any of the data pages is released, the processor updates page management information for a segment including the data page to be released, and writes a batch of data including the updated page management information to the external storage; The storage control device according to claim 1.

4. The processor is configured to perform a write process, which is a process of writing data to the database in response to a write request; In the write process, the processor determines whether a data block containing a data page to which data is to be written is the same as a data block containing a data page that was previously used as a data write destination and is a different data page from the data block; The execution condition is that the result of the determination is false, and in this case, the processor writes to the external storage a batch of data that is a data page different from the data page designated as the data write destination and that includes the data page designated as the data write destination previously. The storage control device according to claim 1.

5. the implementation condition corresponds to an end of the writing process, The end of the write process refers to the issuance of a Commit or the end of an Import process, The import process is completed when all of the data to be imported has been written to the cache device. Upon completion of the write process, the processor: Identifying an unwritten data page corresponding to the write process, the unwritten data page being a data page corresponding to data that has been written to the cache device but not yet written to the external storage; writing a batch of data including data corresponding to the identified unwritten data page and present in the cache device to the external storage; The storage control device according to claim 4.

6. The processor, a first software for causing a processor to function as a DBMS (DataBase Management System) for writing data to the database; second software for causing a processor to function as a Software Defined Storage (SDS) for writing data to the cache device and the external storage; Run The processor executes the second software, providing a logical device, which is a logical storage space, to the DBMS; receiving a write request for the logical device in units of pages from the DBMS; In response to the write request, writing data to the cache device; determining whether the implementation condition is satisfied; If the result of the determination is true, writing the data stored in the cache device to the external storage in a batch unit. The storage control device according to claim 1.

7. The implementation condition is that a logical address that satisfies the following (a) and (b) exists: (a) a logical address within a certain distance in the direction toward smaller addresses from the current write destination logical address, (b) a logical address corresponding to data that exists in the cache device but does not exist in the external storage; The batch data includes a plurality of DBMS write units of data, A DBMS write unit is a unit of data writing from the DBMS to the logical device. The storage control device according to claim 6.

8. The implementation conditions are conditions represented by information provided from the DBMS to the SDS, or conditions represented by information predefined in the SDS. The storage control device according to claim 6.

9. the storage control device is an on-premise device, the cache device is a block device; The storage control device according to claim 1.

10. A method for updating a database by a storage control device that is connected to an external storage device and has a cache device, the method comprising: a first step of writing data to the cache device in units of pages; a second step of determining whether or not an implementation condition for external writing, which is writing to the external storage, is satisfied; a third step of writing data stored in the cache device to the external storage in a batch unit that is a write unit larger than the page unit when the result of the determination is true; having The database is stored in a plurality of pages which are a plurality of logical storage areas constituting a logical storage space; The unit of data writing to the database in the logical storage space is a page unit, The external storage and the collective unit are either of the following (A) and (B), (A) the external storage is an object storage, and the collective unit is an object unit in which all data in one or more segments is treated as an object. (B) the external storage is a file storage, and the collective unit is a file unit in which all data in one or more segments is treated as a file; the storage control device is adapted to write page management information for each segment including a plurality of data pages; For each segment: A data page is a page where database data is stored. The page management information is management information that indicates which data pages in the segment are free. In the third step, in addition to the data in a data page, page management information for a segment including the data page is included in the batch unit of data written to the external storage. Storage control method.

Images